Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF

中文说明

Experimental community conversion. This is not an official Qwen release.

This repository contains a single GGUF text-inference model:

Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf

It is derived from the repaired BF16 model redashes/Qwen3.8-27B-BF16-SSMFIX, which in turn is derived from the official Qwen/Qwen3.8-27B.

At a glance

Property	Value
Model family	Qwen3.8
Parameter count	27B dense model
Artifact format	GGUF V3
Quantization label	UD-Q3_K_XL
Quantization style	Custom mixed quantization with importance matrix
Effective size	12,807.91 MiB, approximately 12.52 GiB
Reported rate	3.93 BPW
Context inherited from upstream	262,144 tokens; practical capacity depends on runtime and hardware
Primary runtime	llama.cpp and compatible GGUF runtimes
Intended modality of this file	Text inference

UD-Q3_K_XL is a quantization-tier name, not a claim that every tensor is stored as pure Q3_K. The exact mixed recipe is documented below and in quantization_recipe.txt.

Model lineage and attribution

Qwen/Qwen3.8-27B
        |
        v
redashes/Qwen3.8-27B-BF16-SSMFIX
        |
        v
Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF

• Upstream model: Qwen/Qwen3.8-27B
• Direct BF16 source: redashes/Qwen3.8-27B-BF16-SSMFIX
• Background discussion: Qwen3.8-27B discussion #76
• GGUF publisher: Luis23333

The direct source model describes itself as an independent verification of a community investigation into abnormal SSM/conv1d scale behavior. This GGUF preserves that repaired source lineage and then applies the quantization recipe described below. It should be treated as an experimental community artifact, not as an official correction to Qwen3.8.

What SSMFIX changes

The BF16 source model applies per-layer alpha rescaling to eight ssm_conv1d.weight tensors. The documented correction factors are:

Layer	Tensor	Alpha
52	blk.52.ssm_conv1d.weight	0.59005
53	blk.53.ssm_conv1d.weight	0.55484
56	blk.56.ssm_conv1d.weight	0.54486
57	blk.57.ssm_conv1d.weight	0.53574
58	blk.58.ssm_conv1d.weight	0.60972
60	blk.60.ssm_conv1d.weight	0.48136
61	blk.61.ssm_conv1d.weight	0.65327
62	blk.62.ssm_conv1d.weight	0.61856

In the local quantization log, ssm_conv1d.weight tensors remain F32 in the GGUF conversion. The repair is therefore not deliberately quantized away by the recipe.

Quantization recipe

The model was quantized from a BF16 GGUF generated from the repaired source weights using llama.cpp build 9222 (9a532ae4b) and an importance matrix named imatrix_unsloth.gguf_file.

Tensor group	Type
Default quantization	IQ3_S
Token embeddings	Q3_K
Output tensor	Q5_K
attn_v	Q5_K
Selected attention, FFN, SSM and MTP tensors	IQ4_XS
ssm_conv1d.weight	F32

The exact tensor overrides are recorded in quantization_recipe.txt. The resulting file is a custom mixed quantization, not a pure single-type Q3_K file.

Important modality limitation

The upstream Qwen3.8 model is a vision-language model. This repository contains only the language-model GGUF produced by the conversion above. It does not include an mmproj/vision projector file, so this single artifact is documented and tested for text inference only. Use the upstream Transformers repository or a compatible model-plus-projector package when image or video input is required.

Usage with llama.cpp

Use a recent llama.cpp build with Qwen3.8/Qwen3.5 architecture support.

Interactive text chat

llama-cli \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning on \
  -cnv

Disable thinking with --reasoning off when supported by your llama.cpp build.

HTTP server

llama-server \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning auto \
  --ctx-size 16384 \
  --n-gpu-layers auto \
  --fit on \
  --flash-attn auto

The 262,144-token context length is inherited metadata, not a guarantee that a particular GPU can allocate that context. A 16 GiB GPU should start with a smaller context such as 16,384 and increase it only after measuring memory use.

The upstream Qwen recommendations are approximately:

• Thinking: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0.
• Non-thinking: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5.

Evaluation and local smoke tests

Reference results from the BF16 source

The following results belong to the direct BF16 source model, not to this GGUF. They are included only as provenance context and should not be interpreted as a GGUF quality benchmark.

Metric	BF16 source v2
MT-Bench average	7.47
IFEval prompt strict	0.5194
IFEval instruction strict	0.6343
GSM8K strict	0.9644
CMMLU	0.6996
TruthfulQA MC1 / MC2	0.3758 / 0.5513
TruthfulQA generation ROUGE-1 / ROUGE-2 / ROUGE-L / BLEU	0.345 / 0.246 / 0.345 / 0.256

GGUF runtime smoke test

Local text-only smoke tests were run with llama.cpp build 9222 (9a532ae4b) on an NVIDIA RTX 5070 Ti, with context size 16,384, six prompts, three runs per prompt, and a maximum of 256 generated tokens.

Mode	Successful requests	Average elapsed time	Average wall generation rate
Baseline	18/18	6506.96 ms	39.367 tokens/s
MTP2	18/18	23098.24 ms	11.180 tokens/s

These are runtime smoke-test measurements, not broad capability scores. In this local setup MTP2 was slower than baseline, so this repository does not claim that speculative decoding is faster for this artifact.

Limitations and disclaimer

• This is an experimental community conversion and is not an official Qwen release.
• The SSMFIX hypothesis and its claimed benefits are not guaranteed for every workload.
• The GGUF has not been assigned the BF16 source's quality scores.
• This upload contains no vision projector and is not presented as a single-file multimodal package.
• Long-context behavior depends on the runtime, KV-cache settings, available VRAM/RAM and prompt shape.
• Validate the model on your own workload before relying on it in production.

License

The model follows the Apache-2.0 license of the upstream Qwen release. See LICENSE. Please preserve the upstream attribution and the direct-source attribution when redistributing this derived artifact.