---
license: apache-2.0
tags:
- qwen
- qwen3.5
- multimodal
- vision
- gguf
- ssm
- quantization
base_model: Qwen/Qwen3.8-27B
---

# Qwen3.8-27B — GGUF quantizations (XYZ mix)

GGUF quantizations of [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)
(multimodal, hybrid SSM + attention, 64 layers + 1 MTP layer), converted with
llama.cpp and quantized with a per-tensor mixed recipe ("XYZ" — importance-based
mixture of quant types tuned to each target bpw, generated by a knapsack
selection over tensor categories).

## Files with MTP

All files below include the built-in **MTP block** (`blk.64.nextn.*`,
`nextn_predict_layers=1`), usable for speculative decoding.

| File | Size (GiB) | BPW (approx) | Target |
|---|---|---|---|
| Qwen3.8-27B-Q3-XYZ.gguf | 10.11 | 3.18 | Q3_K |
| Qwen3.8-27B-Q3.5-XYZ.gguf | 12.01 | 3.78 | Q3.5 |
| Qwen3.8-27B-Q4-XYZ.gguf | 14.15 | 4.45 | Q4_K |
| Qwen3.8-27B-Q4.5-XYZ.gguf | 14.45 | 4.54 | Q4.5 |
| Qwen3.8-27B-Q5-XYZ.gguf | 16.21 | 5.09 | Q5_K |
| Qwen3.8-27B-Q5.5-XYZ.gguf | 18.19 | 5.72 | Q5.5 |
| Qwen3.8-27B-Q6-XYZ.gguf | 21.00 | 6.60 | Q6_K |
| mmproj-Qwen3.8-27B-f32.gguf | 1.72 | — | vision projector (F32) |

## Files without MTP (vanilla)

Same quantizations with the MTP block removed (`nextn_predict_layers=0`,
`block_count=64`, 15 tensors less — slightly smaller).

| File | Size (GiB) | BPW (approx) | Target |
|---|---|---|---|
| Qwen3.8-27B-Q3-vanilla-XYZ.gguf | 9.98 | ~3.14 | Q3_K |
| Qwen3.8-27B-Q3.5-vanilla-XYZ.gguf | 11.85 | ~3.72 | Q3.5 |
| Qwen3.8-27B-Q4-vanilla-XYZ.gguf | 13.97 | ~4.39 | Q4_K |
| Qwen3.8-27B-Q4.5-vanilla-XYZ.gguf | 14.24 | ~4.48 | Q4.5 |
| Qwen3.8-27B-Q5-vanilla-XYZ.gguf | 15.99 | ~5.03 | Q5_K |
| Qwen3.8-27B-Q5.5-vanilla-XYZ.gguf | 17.93 | ~5.64 | Q5.5 |
| Qwen3.8-27B-Q6-vanilla-XYZ.gguf | 20.69 | ~6.50 | Q6_K |

## Usage

With the vision projector (multimodal):

```bash
llama-qwen2vl-cli -m Qwen3.8-27B-Q5-XYZ.gguf --mmproj mmproj-Qwen3.8-27B-f32.gguf
```

With MTP speculative decoding (non-vanilla files):

```bash
llama-cli -m Qwen3.8-27B-Q5-XYZ.gguf --spec-type mtp
```

**Recommended sampling**: `--temp 0.6` for balanced, coherent output.

## Notes

- Model is Apache-2.0, architecture `Qwen3_5ForConditionalGeneration` (hybrid
  SSM + attention, full attention every 4th layer), vocab 248,320,
  `tie_word_embeddings=false` (embedding + output head are Q8_0 in all files).
- Q2/Q2.5 levels are not published: with this vocab, the embedding/output head
  in Q8_0 sets a floor around ~3.1 bpw, so those targets saturate.
- Quantized with llama.cpp (upstream), 4 threads, `--tensor-type` per-tensor
  recipe; sizes are exact file sizes on disk (GiB, base 1024 — matches the
  sizes shown on the HF file browser).