Qwen3.6-27B-MTP-MoQ-GGUF

This repository contains GGUF quantizations of Qwen3.6-27B-MTP using projected MoQ tensor/layer policies. The policies are derived from the Qwen3.5-9B MoQ releases by w-ahmad and adapted to the Qwen3.6-27B MTP architecture.

Source policy reference:

https://huggingface.co/w-ahmad/Qwen3.5-9B-GGUF-MoQ-MTP

All models in this repository were quantized with the unsloth imatrix for Qwen3.6-27B. The MTP block is preserved, and large MTP tensors are explicitly kept at Q8_0.

Available Models

• MoQ-4.8 MTP-Q8_0
• MoQ-4.85 MTP-Q8_0
• MoQ-4.9 MTP-Q8_0
• MoQ-4.95 MTP-Q8_0
• MoQ-5.1 MTP-Q8_0

The 4.85 and 4.95 variants are custom recipes built from the observed 4.8 and 4.9 policies:

• MoQ-4.85: starts from MoQ-4.8, keeps the MoQ-4.8 BF16 guard tensors, and additionally upgrades all MoQ-4.9 extra attn_qkv.weight tensors to Q5_K.
• MoQ-4.95: starts from MoQ-4.9 and restores the MoQ-4.8 ssm_beta.weight BF16 guard tensors.

Policy Extraction Method

The original MoQ method is not publicly documented in full. This repository uses the tensor assignments observable in the published Qwen3.5-9B MoQ GGUF files and projects them to Qwen3.6-27B-MTP.

The extraction pipeline is:

1. Read every tensor name, shape, and GGML tensor type from the Qwen3.5-9B MoQ GGUF files.
2. Split tensor names into layer id and tensor suffix, for example blk.12.attn_qkv.weight -> layer 12 + attn_qkv.weight.
3. Map Qwen3.6-27B layer ids to the source Qwen3.5-9B layer ids by normalized relative depth:

src_layer = round(dst_layer * (src_layers - 1) / (dst_layers - 1))

4. Reuse the source tensor type for matching tensor suffixes.
5. For missing suffixes, fall back to the majority tensor type observed for that suffix.
6. Keep non-matmul and normalization tensors in the source-selected high precision type.
7. Force large MTP tensors in blk.64 to Q8_0:

blk.64.attn_q.weight
blk.64.attn_k.weight
blk.64.attn_v.weight
blk.64.attn_output.weight
blk.64.ffn_up.weight
blk.64.ffn_gate.weight
blk.64.ffn_down.weight
blk.64.nextn.eh_proj.weight

This is a projection of an observed MoQ policy, not the original MoQ optimizer.

Tensor Importance Tiers

The projected policies suggest the following rough sensitivity tiers for Qwen3.5/Qwen3.6 style hybrid MTP models:

Tier	Typical tensor classes	Observed treatment
Critical state / normalization	attn_norm, post_attention_norm, ssm_norm, attn_q_norm, attn_k_norm, ssm_a, ssm_dt.bias, ssm_conv1d.weight	F32
MTP large tensors	blk.64 attention projections, FFN projections, nextn.eh_proj	forced Q8_0
SSM guards	ssm_beta.weight, and in higher BPW also ssm_alpha.weight	BF16 or Q4_K depending on recipe
High-impact projections	ffn_up.weight, selected attn_output.weight, attn_qkv.weight, and in 5.1 also ffn_down.weight	usually Q5_K
Medium-impact projections	attn_k.weight, attn_v.weight, ssm_alpha.weight, ssm_beta.weight in lower recipes	often Q4_K
Lower-risk large tensors	ffn_gate.weight, ffn_down.weight in lower recipes, attn_gate.weight, ssm_out.weight, some query/QKV tensors	often IQ4_XS

Observed custom recipe details:

• MoQ-4.8: BF16 guard is ssm_beta.weight on 48 projected layers.
• MoQ-4.9: upgrades 48 attn_qkv.weight tensors to Q5_K, but removes the 4.8 BF16 guard.
• MoQ-4.85: combines both: 48 attn_qkv.weight tensors at Q5_K plus the 48 ssm_beta.weight BF16 guard tensors.
• MoQ-4.95: starts from 4.9 and restores the 48 ssm_beta.weight BF16 guard tensors.
• MoQ-5.1: further upgrades ffn_down.weight to Q5_K and keeps both ssm_alpha.weight and ssm_beta.weight at BF16.

Usage

Use a recent llama.cpp build with Qwen3.6 MTP support.

Basic generation:

./llama-cli \
  -m Qwen3.6-27B-MTP-MoQ-4.85.gguf \
  -ngl 999 \
  -fa 1 \
  -p "Write a concise technical explanation of transformer inference." \
  -n 256 \
  -st --simple-io

MTP speculative decoding:

./llama-cli \
  -m Qwen3.6-27B-MTP-MoQ-4.85.gguf \
  -ngl 999 \
  -fa 1 \
  -p "Write a concise technical explanation of transformer inference." \
  -n 256 \
  -st --simple-io \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-default

Quantization Method

The models were produced with a customized llama.cpp build that supports --tensor-type-file and imatrix-aware quantization.

Generic command:

./llama-quantize \
  --imatrix imatrix_unsloth.gguf_file \
  --tensor-type-file qwen36_27b_mtp_moq_tensor_types.txt \
  Qwen3.6-27B-MTP-BF16.gguf \
  Qwen3.6-27B-MTP-MoQ.gguf \
  Q4_K \
  16

Notes:

• --tensor-type-file specifies the exact per-tensor recipe.
• Q4_K is the fallback/default quantization type for tensors not matched by the recipe.
• All published models here use the unsloth Qwen3.6-27B imatrix.
• MTP large tensors are explicitly set to Q8_0 in the recipe.

Quality Evaluation

Quality was evaluated on WikiText-2 raw test text with context length 512, using BF16 logits as the KLD base.

Model	BPW	Size GiB	PPL	Mean KLD	Max KLD	99.9% KLD	RMS delta-p	Same top-p
MoQ-4.8 MTP-Q8_0	4.73	15.0410	6.935757	0.023355	23.351997	1.778970	4.163 +/- 0.057%	94.012 +/- 0.062%
MoQ-4.85 MTP-Q8_0	4.84	15.4072	6.946419	0.022245	22.361801	1.801557	4.081 +/- 0.058%	94.208 +/- 0.061%
MoQ-4.9 MTP-Q8_0	4.84	15.3890	6.949457	0.022803	22.039566	2.028698	4.096 +/- 0.057%	94.135 +/- 0.061%
MoQ-4.95 MTP-Q8_0	4.84	15.4048	6.954812	0.022458	22.106335	1.913316	4.046 +/- 0.056%	94.139 +/- 0.061%
MoQ-5.1 MTP-Q8_0	5.11	16.2531	6.920248	0.019191	22.451937	1.720673	3.789 +/- 0.059%	94.594 +/- 0.059%
Unsloth Q4_K_M	5.01	15.9319	6.936849	0.022354	21.246073	2.224004	4.044 +/- 0.060%	94.329 +/- 0.060%

Performance

Throughput was measured on RTX 5090 with a CUDA llama.cpp build. Standard bench uses f16 KV, -ngl 999, -fa 1, -r 3.

Model	pp512 tok/s	tg128 tok/s	pg32768,256 tok/s	MTP p512 prefill	MTP gen128	MTP p32768 prefill	MTP gen256
MoQ-4.8 MTP-Q8_0	2297.23	67.51	1793.82	1428.80	109.60	2270.90	90.10
MoQ-4.85 MTP-Q8_0	2317.31	67.65	1778.50	1312.30	97.90	2241.30	107.60
MoQ-4.9 MTP-Q8_0	2273.29	66.83	1762.49	1459.60	102.30	2247.00	108.80
MoQ-4.95 MTP-Q8_0	2554.09	67.26	1762.94	1403.80	101.80	2244.50	97.40
MoQ-5.1 MTP-Q8_0	2203.42	61.56	1716.57	1391.70	81.20	2209.70	87.50
Unsloth Q4_K_M	2217.93	65.52	1755.85	1265.20	94.80	2171.10	82.00

Tensor Distribution

Model	BF16	F32	IQ4_XS	Q4_K	Q5_K	Q6_K	Q8_0	Other
MoQ-4.8 MTP-Q8_0	48	360	288	80	82	0	8	0
MoQ-4.85 MTP-Q8_0	48	360	240	80	130	0	8	0
MoQ-4.9 MTP-Q8_0	0	360	256	112	130	0	8	0
MoQ-4.95 MTP-Q8_0	48	360	256	64	130	0	8	0
MoQ-5.1 MTP-Q8_0	96	360	176	32	194	0	8	0
Unsloth Q4_K_M	0	0	0	294	48	67	1	456

Practical Recommendation

• MoQ-4.8 is the smallest strong baseline and has the best tail behavior in the lower-size group.
• MoQ-4.85 is the best balanced custom recipe: lower mean KLD than 4.8/4.9 while keeping p99.9 KLD close to 4.8.
• MoQ-4.95 confirms the value of the ssm_beta.weight BF16 guard, but its PPL is slightly worse than 4.85.
• MoQ-5.1 is the highest-quality option in this set, at the cost of larger size and slower throughput.

Notes

• Thanks to the unsloth team for the Qwen3.6 imatrix file.
• Thanks to w-ahmad for publishing the Qwen3.5-9B MoQ GGUF tensor policies.