Jackrong/DeepSeek-V4-Pro-Qwen3.5-9B-MTP-GGUF

🧠 DeepSeek-V4-Pro-Qwen3.5-9B

Distillation Release

A compact 9B reasoning model distilled from DeepSeek-V4-Pro in Max Effect mode

🔬 DeepSeek-V4-Pro Distillation 🧠 9B Parameters 📐 ~250K Math & STEM Samples ⚡ MTP & NVFP4 Evaluated

💡 What is DeepSeek-V4-Pro-Qwen3.5-9B?

DeepSeek-V4-Pro-Qwen3.5-9B is a reasoning-focused fine-tune of Qwen3.5-9B, distilled from responses generated by DeepSeek-V4-Pro in Max Effect mode. The release concentrates its supervised signal on mathematics and STEM problem solving while preserving the efficiency and practical deployability of the 9B parameter class.

🧩 Structured Reasoning Learns rigorous problem decomposition and multi-step solution patterns from a strong teacher.

📐 Math & STEM Focus Trained on approximately 250,000 samples centered on mathematics and scientific reasoning.

🧪 Cross-Domain Transfer Shows small post-training gains in programming and tool use despite receiving no coding-specific SFT data.

⚙️ Practical Deployment Evaluated through vLLM NVFP4 and llama.cpp MTP-Q8_0 deployment paths.

🤝 Collaboration & Training Support

🧪 Hardware Cooperation & Joint Collaboration

This project is built in close collaboration with hardware engineer Kyle Hessling, whose infrastructure, training support, and evaluation assistance helped make this release possible.

👉 Follow his hardware and model-training updates on X / Twitter: @KyleHessling1

🦥 Fine-tuning Framework (Unsloth)

The model training workflow is accelerated and memory-optimized with Unsloth. Special thanks to the Unsloth team for making efficient large-model fine-tuning more accessible.

👉 Documentation and fine-tuning guidance: unsloth.ai/docs

🔬 1. Distillation & Training Data

🧬 From DeepSeek-V4-Pro to a Compact 9B Student

The teacher data for this release was generated with DeepSeek-V4-Pro (Max Effect). Approximately 250,000 mathematics and STEM samples were used for supervised fine-tuning, with an emphasis on structured derivation, domain-aware problem decomposition, and reliable final-answer construction.

Special training note: no coding data was included in this fine-tuning run. Nevertheless, later testing showed a small generalization improvement in programming and tool-calling tasks. This should be interpreted as cross-domain transfer from stronger reasoning structure—not as evidence that the model received direct coding supervision.

📊 2. Benchmark Results

⚡ Performance Snapshot

Strong grade-school mathematics performance and the best average MMLU-Pro result among the tested 9B baselines.

GSM8K 94.50% 4-run average · NVFP4 · vLLM

MMLU-Pro Math 92.40% MTP-Q8_0 GGUF · 8K

MMLU-Pro Average 90.53% Math · Physics · Chemistry

Gain vs Base +0.93 MMLU-Pro average points

2.1 GSM8K

The NVFP4 checkpoint was evaluated on GSM8K four times, and the resulting scores were averaged. Its four-run mean accuracy is 94.50% under vLLM. In the comparison supplied with this release, the 9B student slightly exceeds the reported DeepSeek-V4-Pro teacher result and approaches the Llama-3.3-70B-Instruct reference.

Model	Accuracy
MiMo-V2.5-Pro	99.60%
Llama-3.1-405B-Instruct	96.80%
Llama-3.3-70B-Instruct	94.84%
DeepSeek-V4-Pro-Qwen3.5-9B	94.50%
DeepSeek-V4-Pro	92.60%
DeepSeek-V3	89.30%

Evaluation configuration: DeepSeek-V4-Pro-Qwen3.5-9B-NVFP4 · NVFP4 (FP8 E4M3) · vLLM · four GSM8K test runs, with the final 94.50% reported as their arithmetic mean. The reference scores for the other models shown above are sourced from the openai/gsm8k repository.

2.2 MMLU-Pro: Math, Physics & Chemistry

Each model was evaluated on 500 questions from each of three MMLU-Pro subsets—Math, Physics, and Chemistry—for 1,500 questions per model. In this reported comparison, DeepSeek-V4-Pro-Qwen3.5-9B achieves the highest average at 90.53%. Its compact 4B sibling is now included alongside the Qwen and Claude Mythos-distilled references.

Model	Evaluation Build	Math	Physics	Chemistry	Average
DeepSeek-V4-Pro-Qwen3.5-9B	MTP-Q8_0 GGUF	92.40%	89.40%	89.80%	90.53%
Qwen3.5-9B	MTP-Q8_0 GGUF	90.60%	89.00%	89.20%	89.60%
Claude Mythos-distilled 27B	MTP-Q8 GGUF	88.40%	87.60%	82.60%	86.20%
Claude Mythos-distilled 9B	MTP-Q8_0 GGUF	86.00%	78.60%	78.60%	81.07%
DeepSeek-V4-Pro-Qwen3.5-4B	MTP-Q8 GGUF	80.40%	74.20%	74.80%	76.47%

📈 vs. Qwen3.5-9B
+1.80 points in Math, +0.40 in Physics, +0.60 in Chemistry, and +0.93 average points.

🧪 Evaluation setup
500 questions per subset · 1,500 total per model · llama.cpp. The 9B rows use MTP-Q8_0 GGUF at 8K context, the 4B student uses MTP-Q8 GGUF at 64K context, and Claude Mythos-distilled 27B uses MTP-Q8 GGUF at 32K context.

2.3 Inference Efficiency

⚡ More Correct Answers per Reasoning Token

On the same 1,500-question MMLU-Pro sample, the DeepSeek-distilled model combines the highest accuracy with the lowest token cost per correct answer.

vs. Qwen3.5-9B
36.1% fewer tokens
per correct answer, with +0.93 accuracy points.

vs. Claude Mythos-distilled 9B
22.7% fewer tokens
per correct answer, with +9.47 accuracy points.

Correct answers / 1M tokens
316.8
+56.4% vs. Qwen and +29.3% vs. Claude Mythos-distilled 9B.

Model	Accuracy	Avg. Reasoning Tokens / Question	Median	Tokens / Correct Answer	Correct Answers / 1M Tokens
DeepSeek-V4-Pro-Qwen3.5-9B	90.53%	2,858	676	3,157	316.8
Qwen3.5-9B	89.60%	4,425	3,214	4,938	202.5
Claude Mythos-distilled 9B	81.07%	3,310	639	4,083	244.9

🧪 Token Savings by MMLU-Pro Subject

Subject	vs. Qwen3.5-9B	vs. Claude Mythos-distilled 9B
Math	47.4%	35.6%
Physics	30.8%	10.8%
Chemistry	32.1%	23.3%

📐 GSM8K NVFP4 Reasoning Profile

Average
1,627 tokens

Median
372 tokens

P90
3,042 tokens

Per Correct Answer
1,721 tokens

Computed from 2,638 available raw NVFP4 trajectories.

Metric notes: MMLU-Pro efficiency uses 1,500 questions per model and the supplied composite score for the final correct-answer total. “Tokens per correct answer” is total reasoning tokens divided by correct answers; “correct answers per 1M tokens” is the inverse throughput measure. Lower is better for token-cost metrics; higher is better for accuracy and correct-answer throughput. Comparison deltas follow the supplied composite-result calculations, so recomputing them only from the rounded display values may produce a 0.01-point difference. GSM8K statistics summarize the currently available NVFP4 raw trajectories and are reported separately from the MMLU-Pro MTP-Q8_0 comparison.

🛠️ 3. Generalization Beyond the Training Mix

🔁 A Small but Interesting Transfer Effect

Although the SFT mixture intentionally contained no coding data, post-training tests indicated a small improvement in programming and tool-calling behavior. A plausible interpretation is that mathematics and STEM supervision strengthened reusable skills such as decomposition, constraint tracking, verification, and structured action planning.

This observation is preliminary. It should not be read as a claim that the model is coding-specialized, and the release does not replace dedicated code benchmarks or agent-environment evaluation.

🎯 4. Recommended Uses

• Mathematical problem solving and step-by-step derivation
• Physics, chemistry, and broader STEM question answering
• Structured reasoning and analytical instruction following
• Research on reasoning distillation and cross-domain SFT transfer
• Experimental tool-calling or programming workflows where outputs are independently validated

⚠️ 5. Limitations

• This is an experimental 9B community model and remains subject to hallucinations, reasoning errors, and unstable behavior on difficult or underspecified tasks.
• The reported MMLU-Pro results are based on 500 randomly sampled questions from each of the Math, Physics, and Chemistry subsets, rather than the complete subsets; they are not a comprehensive capability or safety evaluation.
• GSM8K and MMLU-Pro used different inference builds and backends, so their scores should not be used to infer quantization equivalence.
• Programming and tool-use gains were observed without direct coding supervision but are described only as small, preliminary generalization gains.
• Users should independently verify high-stakes mathematical, scientific, or factual outputs.

📚 6. Resources, Acknowledgements & Citation

🌟 Open Research Release

Fine-tuning guide: Jackrong LLM Fine-Tuning Guide

Acknowledgements: Kyle Hessling for hardware collaboration, training support, and evaluation assistance; DeepSeek for the teacher model and reasoning data source; Qwen for the base model family; Unsloth for efficient fine-tuning tooling; and the open-source inference community behind vLLM and llama.cpp.

Citation:

@misc{jackrong_deepseek_v4_pro_qwen35_9b,
  title     = {DeepSeek-V4-Pro-Qwen3.5-9B},
  author    = {Jackrong},
  year      = {2026},
  publisher = {Hugging Face}
}