Capicua25x/Qwen3.6-35B-A3B-DSV4Pro-Thinking-Distill-MXFP4-Vision

Qwen3.6-35B-A3B-DSV4Pro-Thinking-Distill — MXFP4 + Vision (RDNA4 / R9700)

The vision-enabled sibling of Capicua25x/Qwen3.6-35B-A3B-DSV4Pro-Thinking-Distill-MXFP4: the same MXFP4 quant of nerkyor's DeepSeek-V4-Pro thinking distill, with the Qwen3.6 base vision tower grafted back in — so it's text + reasoning + vision + MTP, all on the RDNA4 tcclaviger/vllm-rocm-mxfp4-nvfp4 container.

The distill shipped text-only (nerkyor dropped the vision tower on export). This build restores it. Recipe + tooling: https://github.com/Capicua25x/qwen3.6-mxfp4-rdna4

How image support was added (the vision graft)

nerkyor's distill is Qwen3_5MoeForCausalLM (text-only) — no vision tower. But the base Qwen/Qwen3.6-35B-A3B is multimodal, and its vision tower is architecturally independent of the language model: the vision encoder → merger projects image features to out_hidden_size: 2048, which is exactly the LM's hidden_size. So the tower bolts straight on — the same trick used to graft MTP back:

1. Copy the base's 333 model.visual.* tensors (BF16, ~0.9 GB) from a known-good multimodal build into a new shard.
2. Add model.visual.* to quantization_config.ignore so vLLM loads them as BF16 (the vision tower is never quantized), and keep the multimodal Qwen3_5MoeConfig (with vision_config).
3. Serve without --language-model-only (that flag had been stubbing the tower out), with the base's chat_template + preprocessor_config for image-token handling.

Why it works: the DeepSeek-V4-Pro distill is a light LoRA (text-only reasoning traces); it shifted the LM but didn't break its ability to interpret the base vision tower's embedding space. Verified empirically — fed a generated image (text "VISION 7", a red circle, a blue rectangle), the model read the text, shapes, colors, and spatial positions correctly, while still reasoning. (Vision is not lossless like MTP — a heavier-finetuned LM might fail to ground the embeddings — so this is a measured result, not a guarantee for arbitrary distills.)

Serving (vLLM, 2× R9700, TP2)

vllm serve Capicua25x/Qwen3.6-35B-A3B-DSV4Pro-Thinking-Distill-MXFP4-Vision \
  --tensor-parallel-size 2 --gpu-memory-utilization 0.92 --max-model-len 262144 \
  --enable-prefix-caching --max-num-seqs 64 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Do NOT pass --language-model-only here (that disables vision). Send images via the standard OpenAI image_url content blocks. Thinking-on: temperature=0.6 / top_p=0.95.

Validation — identical to the text-only build, plus vision

Gate	This (vision) build	Text-only sibling
SQL regression	136/137, 0 FAIL	136/137, 0 FAIL
Agent eval (tool-calling, thinking-on)	27/27, 0 FAIL	26/27
Single-stream / ceiling (short)	108.9 tok/s / ~128	107 / ~128
MTP draft acceptance (MTP-3)	~57%	~56%
Image understanding	✅	❌ (none)
Cost of vision	+0.9 GB VRAM, 0 text-perf	—

The vision tower only fires on image input, so text/agentic throughput is unchanged.

Notes

• The "8-bit precision" badge is a HuggingFace artifact of MXFP4's uint8 packing — this model is genuinely 4-bit (config.json → num_bits: 4, mxfp4-pack-quantized).
• Built/tested only on gfx1201 (RDNA4, R9700) with tcclaviger/vllm-rocm-mxfp4-nvfp4.

Credits

Same chain as the text-only build, plus the vision tower from Qwen's base:

• Qwen team (Alibaba) — Qwen3.6-35B-A3B base + the vision tower (Apache-2.0).
• nerkyor / "Lynn" — the DSV4Pro-Thinking distill (the reasoning).
• DeepSeek-AI — DeepSeek-V4-Pro, the distillation teacher.
• olka — qstream, the MXFP4 quantizer.
• kallepahajoki — the qstream Qwen3.6 fix + reference MXFP4 recipe (pahajokiconsulting/Qwen3.6-35B-A3B-MXFP4), the source of the BF16 vision-tower weights grafted here.
• tcclaviger — the RDNA4/gfx12 MXFP4 vLLM container & kernel.

Quantization + RDNA4 packaging (config-wrap, MTP graft, vision graft) by Capicua25x. Apache-2.0.