MidnightPhreaker/Qwen3.6-27B-MTP-TQ4_1S-GGUF

Qwen3.6-27B-MTP TQ4_1S GGUF

This repository contains a TurboQuant/TQ4_1S GGUF quantization of Qwen/Qwen3.6-27B.

This GGUF will not work with original upstream llama.cpp. It requires a TurboQuant-enabled build with TQ4_1S support, such as TheTom/llama-cpp-turboquant.

The source GGUF used for this quantization was Qwen3.6-27B-MTP-BF16.gguf. The final artifact keeps the MTP/next-token-prediction block in BF16 because quantizing that block was found to break MTP behavior.

File

File	Size	SHA256
Qwen3.6-27B-MTP-TQ4_1S.gguf	21,403,947,936 bytes	1e9303f1e00daf4a85ad04fe9f31738526fb00bbafa944b0b68f2fba6aa4440e

Quantization Policy

This model was quantized with a llama.cpp TurboQuant fork that supports TQ4_1S weight tensors.

Workflow:

1. BF16 source GGUF to mixed Q8_0 source GGUF.
2. blk.64 / nextn / MTP tensors preserved as BF16 during the Q8_0 pass.
3. Final tensor-role-aware hybrid TQ4_1S pass from the mixed Q8_0 source.

Final tensor mix:

Tensor type	Count	Notes
TQ4_1S	180	Middle-layer full-attention Q/K/V/O and FFN gate/up tensors
Q4_K	60	Middle-layer FFN down tensors
Q8_0	258	Embeddings, output, SSM/Mamba tensors, protected boundary tensors
BF16	8	blk.64 MTP-critical tensors
F32	360	Norms and small scalar tensors

Protected layout:

• Main text blocks: blk.0 through blk.63
• MTP/nextn block: blk.64
• Protected boundary blocks: blk.0, blk.1, blk.62, blk.63
• Quantized middle blocks: blk.2 through blk.61
• blk.64 is preserved for MTP correctness

The BF16-preserved blk.64 tensors are:

blk.64.attn_k.weight
blk.64.attn_output.weight
blk.64.attn_q.weight
blk.64.attn_v.weight
blk.64.ffn_down.weight
blk.64.ffn_gate.weight
blk.64.ffn_up.weight
blk.64.nextn.eh_proj.weight

The nextn norm tensors remain F32.

Compatibility

This GGUF uses TQ4_1S tensor type support from llama-cpp-turboquant. Standard upstream llama.cpp builds that do not know the TQ4_1S GGUF tensor type may fail to load this file.

Use a build with TQ4_1S weight support, such as the TurboQuant llama.cpp fork.

Example:

./llama-server \
  -m Qwen3.6-27B-MTP-TQ4_1S.gguf \
  -ngl 99 \
  -c 131072 \
  --cache-type-k q8_0 \
  --cache-type-v turbo3

For a more conservative KV cache setting, use --cache-type-k q8_0 --cache-type-v q8_0.

Provenance

• Parent model: Qwen/Qwen3.6-27B
• Source format: BF16 GGUF with MTP tensors
• Quantizer: llama.cpp TurboQuant fork with TQ4_1S support
• Final GGUF: Qwen3.6-27B-MTP-TQ4_1S.gguf

Notes

This is an experimental GGUF quantization. Validate quality and MTP behavior on your workload before production use.