---
license: apache-2.0
language:
- en
- zh
base_model:
- redashes/Qwen3.8-27B-BF16-SSMFIX
base_model_relation: quantized
tags:
- gguf
- qwen3_5
- qwen3.8
- text-generation
- quantized
- imatrix
- iq3_s
- iq4_xs
- q3_k
- q5_k
- ssm-fix
- conv1d-repair
- llama.cpp
---

# Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF

> [中文说明](README_zh.md)

> Experimental community conversion. This is not an official Qwen release.

This repository contains a single GGUF text-inference model:

`Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf`

It is derived from the repaired BF16 model [redashes/Qwen3.8-27B-BF16-SSMFIX](https://huggingface.co/redashes/Qwen3.8-27B-BF16-SSMFIX), which in turn is derived from the official [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B).

## At a glance

| Property | Value |
|---|---|
| Model family | Qwen3.8 |
| Parameter count | 27B dense model |
| Artifact format | GGUF V3 |
| Quantization label | UD-Q3_K_XL |
| Quantization style | Custom mixed quantization with importance matrix |
| Effective size | 12,807.91 MiB, approximately 12.52 GiB |
| Reported rate | 3.93 BPW |
| Context inherited from upstream | 262,144 tokens; practical capacity depends on runtime and hardware |
| Primary runtime | llama.cpp and compatible GGUF runtimes |
| Intended modality of this file | Text inference |

`UD-Q3_K_XL` is a quantization-tier name, not a claim that every tensor is stored as pure `Q3_K`. The exact mixed recipe is documented below and in [quantization_recipe.txt](quantization_recipe.txt).

## Model lineage and attribution

```text
Qwen/Qwen3.8-27B
        |
        v
redashes/Qwen3.8-27B-BF16-SSMFIX
        |
        v
Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF
```

- Upstream model: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)
- Direct BF16 source: [redashes/Qwen3.8-27B-BF16-SSMFIX](https://huggingface.co/redashes/Qwen3.8-27B-BF16-SSMFIX)
- Background discussion: [Qwen3.8-27B discussion #76](https://huggingface.co/Qwen/Qwen3.8-27B/discussions/76)
- GGUF publisher: Luis23333

The direct source model describes itself as an independent verification of a community investigation into abnormal SSM/conv1d scale behavior. This GGUF preserves that repaired source lineage and then applies the quantization recipe described below. It should be treated as an experimental community artifact, not as an official correction to Qwen3.8.

## What SSMFIX changes

The BF16 source model applies per-layer alpha rescaling to eight `ssm_conv1d.weight` tensors. The documented correction factors are:

| Layer | Tensor | Alpha |
|---:|---|---:|
| 52 | `blk.52.ssm_conv1d.weight` | 0.59005 |
| 53 | `blk.53.ssm_conv1d.weight` | 0.55484 |
| 56 | `blk.56.ssm_conv1d.weight` | 0.54486 |
| 57 | `blk.57.ssm_conv1d.weight` | 0.53574 |
| 58 | `blk.58.ssm_conv1d.weight` | 0.60972 |
| 60 | `blk.60.ssm_conv1d.weight` | 0.48136 |
| 61 | `blk.61.ssm_conv1d.weight` | 0.65327 |
| 62 | `blk.62.ssm_conv1d.weight` | 0.61856 |

In the local quantization log, `ssm_conv1d.weight` tensors remain F32 in the GGUF conversion. The repair is therefore not deliberately quantized away by the recipe.

## Quantization recipe

The model was quantized from a BF16 GGUF generated from the repaired source weights using llama.cpp build `9222 (9a532ae4b)` and an importance matrix named `imatrix_unsloth.gguf_file`.

| Tensor group | Type |
|---|---|
| Default quantization | `IQ3_S` |
| Token embeddings | `Q3_K` |
| Output tensor | `Q5_K` |
| `attn_v` | `Q5_K` |
| Selected attention, FFN, SSM and MTP tensors | `IQ4_XS` |
| `ssm_conv1d.weight` | `F32` |

The exact tensor overrides are recorded in [quantization_recipe.txt](quantization_recipe.txt). The resulting file is a custom mixed quantization, not a pure single-type Q3_K file.

## Important modality limitation

The upstream Qwen3.8 model is a vision-language model. This repository contains only the language-model GGUF produced by the conversion above. It does not include an `mmproj`/vision projector file, so this single artifact is documented and tested for text inference only. Use the upstream Transformers repository or a compatible model-plus-projector package when image or video input is required.

## Usage with llama.cpp

Use a recent llama.cpp build with Qwen3.8/Qwen3.5 architecture support.

### Interactive text chat

```bash
llama-cli \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning on \
  -cnv
```

Disable thinking with `--reasoning off` when supported by your llama.cpp build.

### HTTP server

```bash
llama-server \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning auto \
  --ctx-size 16384 \
  --n-gpu-layers auto \
  --fit on \
  --flash-attn auto
```

The 262,144-token context length is inherited metadata, not a guarantee that a particular GPU can allocate that context. A 16 GiB GPU should start with a smaller context such as 16,384 and increase it only after measuring memory use.

The upstream Qwen recommendations are approximately:

- Thinking: `temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`.
- Non-thinking: `temperature=0.7`, `top_p=0.80`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`.

## Evaluation and local smoke tests

### Reference results from the BF16 source

The following results belong to the direct BF16 source model, not to this GGUF. They are included only as provenance context and should not be interpreted as a GGUF quality benchmark.

| Metric | BF16 source v2 |
|---|---:|
| MT-Bench average | 7.47 |
| IFEval prompt strict | 0.5194 |
| IFEval instruction strict | 0.6343 |
| GSM8K strict | 0.9644 |
| CMMLU | 0.6996 |
| TruthfulQA MC1 / MC2 | 0.3758 / 0.5513 |
| TruthfulQA generation ROUGE-1 / ROUGE-2 / ROUGE-L / BLEU | 0.345 / 0.246 / 0.345 / 0.256 |

### GGUF runtime smoke test

Local text-only smoke tests were run with llama.cpp build `9222 (9a532ae4b)` on an NVIDIA RTX 5070 Ti, with context size 16,384, six prompts, three runs per prompt, and a maximum of 256 generated tokens.

| Mode | Successful requests | Average elapsed time | Average wall generation rate |
|---|---:|---:|---:|
| Baseline | 18/18 | 6506.96 ms | 39.367 tokens/s |
| MTP2 | 18/18 | 23098.24 ms | 11.180 tokens/s |

These are runtime smoke-test measurements, not broad capability scores. In this local setup MTP2 was slower than baseline, so this repository does not claim that speculative decoding is faster for this artifact.

## Limitations and disclaimer

- This is an experimental community conversion and is not an official Qwen release.
- The SSMFIX hypothesis and its claimed benefits are not guaranteed for every workload.
- The GGUF has not been assigned the BF16 source's quality scores.
- This upload contains no vision projector and is not presented as a single-file multimodal package.
- Long-context behavior depends on the runtime, KV-cache settings, available VRAM/RAM and prompt shape.
- Validate the model on your own workload before relying on it in production.

## License

The model follows the Apache-2.0 license of the upstream Qwen release. See [LICENSE](LICENSE). Please preserve the upstream attribution and the direct-source attribution when redistributing this derived artifact.