---
license: apache-2.0
tags:
- qwen
- qwen3.5
- multimodal
- vision
- gguf
- ssm
- quantization
base_model: Qwen/Qwen3.8-27B
---

# Qwen3.8-27B — GGUF quantizations (XYZ mix)

GGUF quantizations of [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)
(multimodal, hybrid SSM + attention, 64 layers + 1 MTP layer), converted with
llama.cpp and quantized with a per-tensor mixed recipe ("XYZ" — importance-based
mixture of quant types tuned to each target bpw, generated by a knapsack
selection over tensor categories).

## Files (with MTP)

All files below include the built-in **MTP block** (`blk.64.nextn.*`,
`nextn_predict_layers=1`), usable for speculative decoding.

| File | Size | BPW (approx) | Target |
|---|---|---|---|
| Qwen3.8-27B-Q3-XYZ.gguf | 10.85 GB | 3.18 | Q3_K |
| Qwen3.8-27B-Q3.5-XYZ.gguf | 12.90 GB | 3.78 | Q3.5 |
| Qwen3.8-27B-Q4-XYZ.gguf | 15.19 GB | 4.45 | Q4_K |
| Qwen3.8-27B-Q4.5-XYZ.gguf | 15.52 GB | 4.54 | Q4.5 |
| Qwen3.8-27B-Q5-XYZ.gguf | 17.41 GB | 5.09 | Q5_K |
| Qwen3.8-27B-Q5.5-XYZ.gguf | 19.53 GB | 5.72 | Q5.5 |
| Qwen3.8-27B-Q6-XYZ.gguf | 22.55 GB | 6.60 | Q6_K |
| mmproj-Qwen3.8-27B-f32.gguf | 1.84 GB | — | vision projector (F32) |

## Files without MTP (vanilla)

Versions without the MTP block (`-vanilla-XYZ.gguf`, `nextn_predict_layers=0`,
slightly smaller) — to be added.

## Usage

With the vision projector (multimodal):

```bash
llama-qwen2vl-cli -m Qwen3.8-27B-Q5-XYZ.gguf --mmproj mmproj-Qwen3.8-27B-f32.gguf
```

With MTP speculative decoding:

```bash
llama-cli -m Qwen3.8-27B-Q5-XYZ.gguf --spec-type mtp
```

## Notes

- Model is Apache-2.0, architecture `Qwen3_5ForConditionalGeneration` (hybrid
  SSM + attention, full attention every 4th layer), vocab 248,320,
  `tie_word_embeddings=false` (embedding + output head are Q8_0 in all files).
- Q2/Q2.5 levels are not published: with this vocab, the embedding/output head
  in Q8_0 sets a floor around ~3.1 bpw, so those targets saturate.
- Quantized with llama.cpp (upstream), 4 threads, `--tensor-type` per-tensor
  recipe; sizes are real file sizes.