Jackrong/DeepSeek-V4-Pro-Qwen3.5-4B-MTP-GGUF

🧠 DeepSeek-V4-Pro-Qwen3.5-4B

Distillation Release

A lightweight 4B reasoning model distilled from DeepSeek-V4-Pro in Max Effect mode

🔬 DeepSeek-V4-Pro Distillation 🧠 4B Parameters 📐 ~250K Math & STEM Samples ⚡ MTP-Q8 Evaluated

💡 What is DeepSeek-V4-Pro-Qwen3.5-4B?

DeepSeek-V4-Pro-Qwen3.5-4B is a reasoning-focused fine-tune of Qwen3.5-4B, distilled from responses generated by DeepSeek-V4-Pro in Max Effect mode. It follows the same training pipeline and approximately 250,000-sample mathematics and STEM mixture used for the 9B release, while targeting a substantially smaller and more accessible deployment class.

🧩 Structured Reasoning Learns multi-step solution patterns and explicit problem decomposition from a strong teacher.

📐 Math & STEM Focus Trained on approximately 250,000 samples centered on mathematical and scientific reasoning.

⚡ Compact Deployment Brings the DeepSeek-distilled training recipe into the efficient 4B parameter class.

🧪 Reproducible Evaluation Evaluated on complete GSM8K runs and fixed MMLU-Pro Math, Physics, and Chemistry samples.

🤝 Collaboration & Training Support

🧪 Hardware Cooperation & Joint Collaboration

This project is built in close collaboration with hardware engineer Kyle Hessling, whose infrastructure, training support, and evaluation assistance helped make this release possible.

👉 Follow his hardware and model-training updates on X / Twitter: @KyleHessling1

🦥 Fine-tuning Framework (Unsloth)

The model training workflow is accelerated and memory-optimized with Unsloth. Special thanks to the Unsloth team for making efficient large-model fine-tuning more accessible.

👉 Documentation and fine-tuning guidance: unsloth.ai/docs

🔬 1. Distillation & Training Data

🧬 The 9B Training Recipe, Compressed into a 4B Student

The teacher data for this release was generated with DeepSeek-V4-Pro (Max Effect). The 4B student uses the same training flow and approximately 250,000 mathematics and STEM samples as DeepSeek-V4-Pro-Qwen3.5-9B, emphasizing structured derivation, problem decomposition, verification, and reliable final-answer construction.

Training scope: no coding data was included in this SFT mixture. The published benchmarks in this card therefore focus on mathematics and STEM; coding and tool-use transfer have not yet been established for the 4B release.

1 · Teacher Generation
DeepSeek-V4-Pro in Max Effect mode produces detailed mathematics and STEM solutions.

2 · Data Preparation
Approximately 250K samples are cleaned and formatted for supervised reasoning transfer.

3 · Unsloth SFT
Qwen3.5-4B is fine-tuned through an Unsloth LoRA pipeline and merged into BF16.

4 · Deployment Build
An MTP-enabled Q8 GGUF build is used for the reported llama.cpp evaluations.

📊 2. Benchmark Results

⚡ Performance Snapshot

Strong grade-school mathematics performance from a compact 4B model, with completed 1,500-question MMLU-Pro STEM results.

GSM8K 91.77% 2-run pass@1 mean

MMLU-Pro Math 80.40% 402 / 500

MMLU-Pro Physics 74.20% 371 / 500

MMLU-Pro Chemistry 74.80% 374 / 500

MMLU-Pro Average 76.47% 1,147 / 1,500

2.1 GSM8K

The MTP-Q8 build was evaluated twice on the complete 1,319-question GSM8K test split. Across 2,638 sampled answers, the model produced 2,421 exact-answer matches, yielding a two-run pass@1 mean of 91.7741%. The two independent run scores were 91.6603% and 91.8878%, a spread of only 0.23 percentage points.

Model	Accuracy
MiMo-V2.5-Pro	99.60%
Llama-3.1-405B-Instruct	96.80%
Llama-3.3-70B-Instruct	94.84%
DeepSeek-V4-Pro-Qwen3.5-9B	94.50%
DeepSeek-V4-Pro	92.60%
DeepSeek-V4-Pro-Qwen3.5-4B	91.77%
DeepSeek-V3	89.30%

📈 Compact-model result
The 4B student is 2.73 points behind the published 9B score while remaining 2.47 points above the DeepSeek-V3 reference shown in the 9B release card.

🧪 4B evaluation setup
1,319 questions × 2 runs · MTP-Q8 GGUF · llama.cpp · 64K context per slot · MTP draft n=2.

Comparison note: the 4B result is a two-run MTP-Q8 GGUF evaluation under llama.cpp, while the published 9B result was measured with an NVFP4 checkpoint under vLLM. Both use the GSM8K test split, but this table is a reported-score comparison rather than a perfectly controlled parameter-scaling experiment. The non-4B reference scores are reproduced from the 9B release card.

2.2 4B vs. 9B

Both students use the same DeepSeek-V4-Pro teacher pipeline and approximately 250,000-sample mathematics and STEM mixture. The 4B release prioritizes local efficiency; the 9B release retains more capacity for difficult multi-step STEM reasoning.

Metric	DeepSeek-V4-Pro-Qwen3.5-4B	DeepSeek-V4-Pro-Qwen3.5-9B
GSM8K	91.77%	94.50%
MMLU-Pro Math	80.40%	92.40%
MMLU-Pro Physics	74.20%	89.40%
MMLU-Pro Chemistry	74.80%	89.80%
MMLU-Pro Average	76.47%	90.53%

Interpretation: the 4B student remains strong on GSM8K, while Math is its strongest subject in the selected MMLU-Pro STEM sample. The 9B student retains higher scores across all three MMLU-Pro subjects. GSM8K and the reported 9B results use different inference builds, so this table should be read as a release overview rather than a controlled scaling experiment.

2.3 MMLU-Pro: Math, Physics & Chemistry

Each model in the comparison below was evaluated on 500 questions from each of three MMLU-Pro subsets—Math, Physics, and Chemistry—for 1,500 questions per model. The table follows the same column design and subject order as the 9B release card. The newly added Claude Mythos-distilled 27B result reaches 86.20% overall and appears directly before Claude Mythos-distilled 9B.

Model	Evaluation Build	Math	Physics	Chemistry	Average
DeepSeek-V4-Pro-Qwen3.5-4B	MTP-Q8 GGUF	80.40%	74.20%	74.80%	76.47%
DeepSeek-V4-Pro-Qwen3.5-9B	MTP-Q8_0 GGUF	92.40%	89.40%	89.80%	90.53%
Qwen3.5-9B	MTP-Q8_0 GGUF	90.60%	89.00%	89.20%	89.60%
Claude Mythos-distilled 27B	MTP-Q8 GGUF	88.40%	87.60%	82.60%	86.20%
Claude Mythos-distilled 9B	MTP-Q8_0 GGUF	86.00%	78.60%	78.60%	81.07%

Comparison note: all rows use 500 questions per subject, but the reported deployment configurations differ. The 4B run uses MTP-Q8 GGUF at 64K context, the 9B comparison rows use MTP-Q8_0 GGUF at 8K context, and Claude Mythos-distilled 27B uses MTP-Q8 GGUF at 32K context under llama.cpp.

⚙️ 3. Deployment Profile

🚀 Built for Practical Local Reasoning

The training adapter was merged with unsloth/Qwen3.5-4B in BF16. The reported deployment checkpoint is an MTP-enabled Q8 GGUF build using two draft tokens under llama.cpp speculative decoding.

The 4B scale is intended for users who value lower memory use and local inference accessibility. Exact speed and memory consumption depend on quantization, hardware, context length, batch size, and runtime configuration; no cross-hardware speed claim is made in this release.

🎯 4. Recommended Uses

• Grade-school and general mathematical problem solving
• Lightweight physics, chemistry, and broader STEM question answering
• Structured reasoning and analytical instruction following
• Local experiments with reasoning distillation and compact models
• MTP-enabled llama.cpp deployment research

⚠️ 5. Limitations

• This is an experimental 4B community model and remains subject to hallucinations, arithmetic mistakes, reasoning failures, and unstable behavior on difficult or underspecified tasks.
• The reported MMLU-Pro result covers fixed 500-question samples from Math, Physics, and Chemistry rather than the complete subsets; broader categories remain unevaluated.
• The GSM8K comparison combines reported results from different inference builds and backends; it is useful as a release overview, not as a perfectly controlled scaling study.
• The model received no coding-specific SFT data, and coding or tool-use generalization has not yet been established for this 4B checkpoint.
• A smaller parameter budget creates a visible gap versus the 9B release on the available MMLU-Pro STEM subsets.
• Users should independently verify high-stakes mathematical, scientific, medical, legal, or factual outputs.

📚 6. Resources, Acknowledgements & Citation

🌟 Open Research Release

Fine-tuning guide: Jackrong LLM Fine-Tuning Guide

Related model: DeepSeek-V4-Pro-Qwen3.5-9B

Acknowledgements: Kyle Hessling for hardware collaboration, training support, and evaluation assistance; DeepSeek for the teacher model and reasoning data source; Qwen for the base model family; Unsloth for efficient fine-tuning and conversion tooling; and the open-source inference community behind llama.cpp.

Citation:

@misc{jackrong_deepseek_v4_pro_qwen35_4b,
  title     = {DeepSeek-V4-Pro-Qwen3.5-4B},
  author    = {Jackrong},
  year      = {2026},
  publisher = {Hugging Face}
}