Qwen3.6-28B-REAP-mixedbit-tuned-mlx

An MLX "mixedbit" quantization of 0xSero/Qwen3.6-28B (a REAP-pruned Qwen3.5/3.6 hybrid MoE), shipped with the buun hardened chat template and a documented, tested inference config.

The weights are not fine-tuned. This repo's value is: (1) the mixed-bit MLX quant (~13 GB), (2) the bundled hardened chat template, and (3) a battle-tested deployment config — see below.

What is "mixedbit"?

A non-uniform MLX quantization: different layers are stored at different bit-widths instead of a single uniform 4-bit. The result here is ~13 GB on disk — actually smaller than a standard uniform 4-bit quant of the same model (~16–17 GB), because some layers are pushed below 4 bits while sensitive layers are kept higher. It runs well on Apple Silicon via mlx-lm / oMLX.

Architecture: qwen3_5_moe — a hybrid linear-attention + MoE model (40 layers, fused 3-D expert tensors, ~3B active params per token). This matters for tooling (see "Known incompatibilities").

Lineage & credits

• Base / weights: 0xSero/Qwen3.6-28B — REAP 20%-pruned from Qwen/Qwen3.6-35B-A3B, LoRA-SFT'd on Claude Opus reasoning traces. All model capability is theirs. This repo only re-quantizes those weights to MLX mixedbit.
• Chat template: spiritbuun/buun-Qwen3.6-chat_template — hardened Qwen3.5/3.6 template (fixes the official template's "empty-think poisoning" bug). The original stock template is preserved here as chat_template.STOCK.jinja.bak.
• Base architecture: Qwen3.5/3.6 by Alibaba/Qwen.

Please honor the licenses/terms of the upstream 0xSero/Qwen3.6-28B and Qwen base models. (No explicit license file was present in the source; defer to upstream.)

Recommended config (tested)

Setting	Value	Why
enable_thinking	False for coding/tool-use/chat	Far more reliable & faster; thinking-ON can ramble.
Thinking budget	~6000 (when thinking ON)	Caps the reasoning loop so it commits instead of spiraling.
Sampling	temp 0.3–0.4, top_p 0.9, top_k 20	(temp ~0.7 for open chat)
fix_mistral_regex	True	Required for this quant's tokenizer.

Two good modes: (1) coding/tool-use → enable_thinking=False (fast, reliable); (2) hard open-ended reasoning → thinking ON + max_tokens ~5000–6000.

⚠️ Known incompatibilities (oMLX speculative features)

Because of the hybrid linear-attention/MoE architecture, these crash and should be left OFF:

• DFlash → [convert] Only length-1 arrays can be converted to Python scalars (in target_qwen_gdn). Also single-stream.
• SpecPrefill → '_OffsetAdjustedRoPE' object has no attribute 'dims'.
• MTP → this quant does not carry the mtp.* tensors.

Plain (non-speculative) decoding works great: ~78 tok/s in chat, ~49 tok/s on large (27K-token) agentic prompts. TurboQuant KV cache @ 6-bit is fine if you want to save runtime memory on long contexts.

Capability (informal, correctly-prompted)

Strong at: math, everyday programming, algorithms (LIS, Dijkstra, DP, thread-safe queues), tool-calling (clean JSON), and agentic planning. Weakness: niche factual recall (e.g. specific geography) — it can hallucinate and ruminate when it lacks knowledge; pair it with a search tool for fact-lookups.