---
language:
  - en
  - zh
base_model:
  - Qwen/Qwen3.6-27B
tags:
  - gguf
  - llama.cpp
  - qwen
  - qwen3
  - qwen3.6
  - mtp
  - speculative-decoding
  - quantized
  - long-context
  - chinese
  - distillation
  - agent
  - react
pipeline_tag: text-generation
license: apache-2.0
library_name: gguf
inference:
  parameters:
    temperature: 0.6
    top_p: 0.95
model-index:
  - name: Opus-DeepSeek-Distilled-Q4M
    results:
      - task:
          type: text-generation
        dataset:
          type: benchlocal
          name: BenchLocal 6-pack
        metrics:
          - type: benchlocal-score
            value: 86.5
            name: 6-pack Total
      - task:
          type: question-answering
        dataset:
          type: gpqa
          name: GPQA-Diamond-198
        metrics:
          - type: accuracy
            value: 83.84
            name: Accuracy
      - task:
          type: question-answering
        dataset:
          type: mmlu
          name: MMLU-500 (5-shot)
        metrics:
          - type: accuracy
            value: 91.80
            name: Accuracy
---

# 🧠 Opus-DeepSeek-Distilled-Q4M

> A distilled Qwen3.6-27B GGUF optimized for local agentic reasoning, tool use, and long-chain task execution.

[![BenchLocal](https://img.shields.io/badge/BenchLocal-86.5-8A2BE2)](https://github.com/brianyin/BenchLocal)
[![GPQA](https://img.shields.io/badge/GPQA--Diamond--198-83.84%25-blue)](https://github.com/brianyin/BenchLocal)
[![MMLU](https://img.shields.io/badge/MMLU--500-91.80%25-green)](https://github.com/brianyin/BenchLocal)
[![License](https://img.shields.io/badge/License-Apache--2.0-orange)](https://www.apache.org/licenses/LICENSE-2.0)
[![GGUF](https://img.shields.io/badge/Format-GGUF-ff69b4)](https://github.com/ggml-org/llama.cpp)

---

## ⚠️ Sampling Parameters

> **Please ensure `temperature = 0.6` and `top_p = 0.95` when using this model.**
>
> This model was trained and validated at these specific parameters. Both too high and too low temperatures cause problems:
>
> 🔥 **Too high (> 0.6)** → Output becomes divergent as token probabilities flatten. This causes malformed tool calls, function name hallucinations, unstable parameter generation, and uncontrollable agent behavior.
>
> 🧊 **Too low (< 0.3)** → The model almost always picks the highest-probability token. In tool-calling scenarios this manifests as: repeating the same failed tool call instead of trying alternatives, shortened thinking traces leading to insufficient reasoning depth, and reduced robustness to edge cases.
>
> This is an **important recommendation** based on extensive real-world testing. Please verify these parameters in your inference framework.

---

## 📢 Highlights

| Area | Score | vs Qwen3.6-27B q4_k_m |
|------|-------|----------------------|
| **BenchLocal 6-pack** 🏆 | **86.5** | **+8.3** |
| **GPQA-Diamond-198** 🔬 | **83.84%** | +10.14% |
| **BugFind-15** 🐛 | **80** | **+20** |
| **ToolCall-15** 🔧 | **97** | +4 |
| **InstructFollow-15** 📋 | **94** | **+17** |
| **StructOutput-15** 📊 | **88** | **+11** |
| **MMLU-500 (5-shot)** 📚 | **91.80%** | ~tied (+0.2%) |
| **DataExtract-15** 📄 | 81 | -2 |

> 🚀 **Output speed**: `~60 tok/s` on A100 40GB · `~100 tok/s` on RTX PRO 6000 (q4_k_m + mtp=3)

---

## 🔥 Why This Model?

The original `Qwen3.6-27B` has solid foundational capabilities, but its **agent behavior falls short** — prone to infinite loops when thinking, lacks structured agent design, and has room to improve in math reasoning.

This variant tackles all three through targeted distillation:

1. ✅ **Infinite loops → Eliminated.** ReAct-style reasoning-action orchestration fixes the root cause.
2. ✅ **Agent behavior → Structured.** Distilled Claude Opus's systematic thinking and organization.
3. ✅ **Math reasoning → Strengthened.** Absorbed capabilities from strong math/logic models.

The result is a local agent that doesn't just score high on benchmarks — it *works* reliably in real engineering tasks like BugFind, where it now competes with GLM5.2.

> ⚠️ **Note**: Side-by-side GLM5.2 comparisons were evaluated using Opus 4.8 as a judge. Opus-as-judge has inherent biases — results are indicative, not definitive.

---

## 🎯 Design Philosophy

**Core insight**: `Qwen3.6-27B` doesn't lack capability — it lacks *good agent behavior*. That makes it worth iterating on.

We followed the **[ReAct](https://arxiv.org/abs/2210.03629)** paradigm (Yao et al., ICLR 2023) — unifying reasoning and action into an alternating, constrained, executable loop — rather than just making the model "think longer" or "call tools better."

| Teacher Model | Capability Distilled |
|---------------|---------------------|
| **Claude Opus** 🎯 | Systematic thinking, structured organization, concise reasoning |
| **DeepSeek** 🧭 | Stable agent behavior, tool orchestration, task closure |
| **Math/Logic models** ➗ | Mathematical reasoning, logical deduction |

---

## 📊 Performance

### BenchLocal 6-pack

| Pack | `q4_k_m` (ours) | `Qwen/Qwen3.6-27B q4_k_m` | Delta |
|------|:-:|:-:|:-:|
| **BugFind-15** 🐛 | **80** | 60 | **+20** |
| **ToolCall-15** 🔧 | **97** | 93 | +4 |
| **DataExtract-15** 📄 | 81 | **83** | -2 |
| **InstructFollow-15** 📋 | **94** | 77 | **+17** |
| **ReasonMath-15** ➗ | 79 | 79 | 0 |
| **StructOutput-15** 📊 | **88** | 77 | **+11** |
| **Total** 🏆 | **86.5** | 78.2 | **+8.3** |

### Extended Evals

| Benchmark | Ours | Baseline | Notes |
|-----------|:----:|:--------:|-------|
| **GPQA-Diamond-198** 🔬 | **83.84%** | 73.7% | +10.14%, all 198 graded locally |
| **MMLU-500 (5-shot)** 📚 | **91.80%** | 91.6% | Approximately tied |

---

## 🛠️ Usage

### Recommended Stack

```
🧩 OpenCode + LM Studio
📐 Temperature: 0.6 · Top-p: 0.95
⚡ q4_k_m + mtp=3
```

### Quick Start (llama.cpp)

```bash
# Download the GGUF
huggingface-cli download your-org/Opus-DeepSeek-Distilled-Q4M \
  opus-deepseek-distilled-q4m-q4_k_m.gguf --local-dir ./models

# Run with llama.cpp
./llama-cli -m ./models/opus-deepseek-distilled-q4m-q4_k_m.gguf \
  --temp 0.6 --top-p 0.95 \
  -p "Your prompt here"
```

### Via LM Studio

1. Load the GGUF file in LM Studio
2. Set backend to llama.cpp
3. Enable MTP (set depth=3) under inference options
4. Set `temperature = 0.6`, `top_p = 0.95`
5. Start the local API server
6. Connect via OpenCode or any OpenAI-compatible client

> 💡 **Pro tip**: For coding tasks, the `temp 0.6 / top_p 0.95` combo delivers the best balance of creativity and correctness.

---

## ⚠️ Known Limitations

### Evaluation Methodology
- **Opus-as-judge biases**: GLM5.2 comparisons are judge-evaluated, not absolute rankings
- **MMLU**: Single-run 5-shot result; variations in shot selection may cause fluctuation

### Capability Boundaries
- **DataExtract**: Scores 81 vs 83 baseline — extraction tasks may have slight regression from distillation
- **Closed-source teachers**: Risks include inherited biases and TOS compliance — assess for your use case
- **27B scale ceiling**: May still hit capacity limits on extremely complex long-chain reasoning

### Deployment Notes
- **MTP=3**: Boosts throughput but adds VRAM overhead — disable or reduce on <24GB hardware
- **Quantization**: Only `q4_k_m` tested; `q5_k_m` may improve accuracy at higher VRAM cost
- **Imatrix**: Uses importance-matrix quantization, not standard k-quant — better parameter preservation at low bit widths

---

## 📝 Citation

```bibtex
@misc{opus-deepseek-distilled-q4m,
  title = {Opus-DeepSeek-Distilled-Q4M: A Distilled Agentic GGUF for Local Deployment},
  author = {Yin, Brian and BenchLocal Contributors},
  year = {2026},
  url = {https://github.com/brianyin/BenchLocal}
}
```

---

## 🙏 Acknowledgements

- **[Qwen](https://github.com/QwenLM/Qwen)** team — excellent foundational model
- **[Unsloth](https://github.com/unslothai/unsloth)** — efficient training infrastructure
- **[Merkyor](https://github.com/Merkyor)** — identified ReAct as the key to solving agent infinite loops
- **Community** — built on existing open-source exploration and practical experience

> *It is because of this work that came before that we can continue pushing forward, arriving at today's more stable, more practical, and more complete agent — and helping us get closer to the era of local agent AI.*