Instructions to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Ollama
How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with Ollama:
```
ollama run hf.co/Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M
```

Unsloth Studio

How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF to start chatting

How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M
```

Lemonade

How to use Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Brian6145/Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.6-27B-Claude-Opus-DeepSeek-Distilled-Imatrix-MTP-GGUF-Q4_K_M

List all available models

lemonade list

Brian6145 commited on 24 days ago

Commit

1ad4ff7

verified ·

1 Parent(s): 9c24f34

Update README.md

Browse files

Files changed (1) hide show

README.md +114 -134

README.md CHANGED Viewed

@@ -31,8 +31,8 @@ model-index:
       - task:
           type: text-generation
         dataset:
-          name: BenchLocal 6-pack
           type: benchlocal
         metrics:
           - type: benchlocal-score
             value: 86.5
@@ -40,8 +40,8 @@ model-index:
       - task:
           type: question-answering
         dataset:
-          name: GPQA-Diamond-198
           type: gpqa
         metrics:
           - type: accuracy
             value: 83.84
@@ -49,188 +49,168 @@ model-index:
       - task:
           type: question-answering
         dataset:
-          name: MMLU-500 (5-shot)
           type: mmlu
         metrics:
           - type: accuracy
             value: 91.80
             name: Accuracy
 ---
-## Summary
-`Opus-DeepSeek-Distilled-Q4M` is a local model build designed for high-intensity reasoning, engineering tasks, and competitive benchmark performance.
-Quick highlights:
-- `BenchLocal 6-pack` scores: `BugFind-15 = 80`, `ToolCall-15 = 97`, `DataExtract-15 = 81`, `InstructFollow-15 = 94`, `ReasonMath-15 = 79`, `StructOutput-15 = 88`
-- `GPQA-Diamond-198` result: `166 / 198 = 83.84%`
-- All 198 GPQA-Diamond questions fully scored and graded locally — no drops, no scoring failures
-- `MMLU-500 (5-shot)` accuracy: `91.80%` (completed)
-- With `q4_k_m + mtp=3` deployment, achieves `~60 tok/s` on A100 40GB and `~100 tok/s` on RTX PRO 6000, approaching API-level fluency
-- In real-world `BugFind` scenarios, matches `GLM5.2` on certain tasks
-- In a side-by-side comparison, `Opus 4.8` as a judge model rated this variant higher than `GLM5.2` on the evaluated tasks (Opus-as-judge has inherent biases — findings are indicative, not definitive)
-This release is not "a new version for the sake of a new version." It emerged from a rigorous model selection and agent design trade-off analysis. After systematically evaluating the current crop of `Qwen3.6` fine-tuned variants, we found that the `35B` series tends to exhibit more pronounced agent stability issues — inconsistent reasoning depth, over-reliance on frequent tool calls, and instability in complex task chains. The `27B` series presents the opposite picture: its foundational capabilities and stability are sufficient, but its **agent behavior patterns are suboptimal and prone to infinite loops**.
-Specifically, the original `Qwen/Qwen3.6-27B` and similar variants exhibit three main problems in agent scenarios:
-1. Enabling thinking mode leads to prolonged self-loops, sometimes infinite loops.
-2. Lack of a more complete, restrained, and systematic agent design style reminiscent of Claude.
-3. Mathematical reasoning, while usable, still has room for improvement.
-The first problem is the hardest — it cannot be fixed with a single prompt tweak. The very orchestration of reasoning and action needs to be re-learned.
-To address this, the current version incorporates specifically enhanced high-quality training data that fundamentally solves the infinite-loop problem at the level of reasoning-action orchestration. After extensive real-world agent usage across long task chains, this issue is now **completely eliminated**. At the same time, these enhancements push the model's reasoning ceiling higher while retaining its agent usability.
-The model's value extends beyond high `BenchLocal` scores — it has already demonstrated competitive performance on longer, harder, and broader benchmarks like `GPQA-Diamond-198` and `MMLU-500 (5-shot)`.
-More importantly, this isn't just paper performance. After sustained local usage in real `BugFind` scenarios, its results have far exceeded conventional expectations for the `Qwen3.6-27B` line. In some cases, it matches `GLM5.2` — something that seemed almost unimaginable before. In a side-by-side comparison where both `GLM5.2` and this model analyzed the same problem, `Opus 4.8` as a judge rated this model higher (Opus-as-judge biases apply — results are indicative, not definitive).
-## Design Background
-The core judgment driving this design: `Qwen3.6-27B` doesn't lack capability — its agent behavior patterns aren't good enough yet. This makes it far more worth iterating on than models with weaker foundations. Our goal isn't simply to make it "better at conversation" but to push it toward "being a more reliable agent."
-The `ReAct` framework provided crucial inspiration. `ReAct (Reason + Act)` by `Yao et al., 2022, arXiv:2210.03629, ICLR 2023` doesn't emphasize reasoning or tool use in isolation — it unifies "reasoning" and "acting" into an alternating, constrained, executable process. Extensive version comparisons and empirical observations confirm this is the key to solving the current problems.
-Accordingly, this version makes explicit design trade-offs:
-- We retain `Qwen3.6-27B`'s already-strong foundational capabilities and stable base.
-- Through real distillation trajectories, we progressively teach it to think, decide, call tools, and close tasks more effectively.
-- We distill `Claude Opus`'s tighter, more systematic thinking and organization skills.
-- We distill `DeepSeek`'s agent-scenario thinking patterns, training the model to behave more like an agent and less like a pure Q&A bot.
-- We also absorb core capabilities from strong mathematical/logical reasoning models to further enhance its math and logical reasoning.
-The result is neither a model that's "just better at tool use" nor one that "just thinks longer" — it's a `Qwen3.6-27B` variant that learns core capabilities from multiple strong models and converges into a better, more practical local agent. It inherits Claude's structured thinking, DeepSeek's stable agent behavior, and strong math models' reasoning ability — unified into a more complete, more usable local agent.
-## Highlights
-- `GPQA-Diamond-198`: `166 / 198 = 83.84%`
-- All 198 GPQA-Diamond questions fully scored and graded locally — no drops, no scoring failures
-- `MMLU-500 (5-shot)`: `91.80%` (completed)
-- Competitive scientific reasoning + broad knowledge coverage
-- Top tier on the `BenchLocal` historical leaderboard
-- Real-world `BugFind` performance entering a highly competitive range
-## Why This Variant Matters
-What makes this quantized build interesting is that it doesn't just perform "adequately" in a local pipeline — it clearly enters first-tier discussions.
-The extended eval results already tell the story:
-- It doesn't just look good on engineering tasks (`BenchLocal 6-pack`)
-- It holds up on hard science (`GPQA`)
-- It's equally strong on broad-spectrum knowledge (`MMLU 5-shot`)
-- It represents a clear methodological direction: equipping 27B-level models with mature agent behavior instead of letting them spiral into inefficient thinking and runaway tool calls
-- Through sustained real-world use, it enters the conversation of "can deliver in real engineering scenarios," especially on `BugFind` — tasks that were never a `Qwen3.6-27B` strength
-This sets it apart from models that score high on one benchmark but fall apart on another.
-## Performance Snapshot
-### BenchLocal Historical Snapshot
-Based on the repository's historical benchmarks, this `q4_k_m` variant ranks among the highest locally.
-| Pack | `q4_k_m` (this variant) | `Qwen/Qwen3.6-27B q4_k_m` | Delta | Notes |
-| --- | --- | --- | --- | --- |
-| `BugFind-15` | `80` | `60` | `+20` | Significant improvement in debugging and issue localization |
-| `ToolCall-15` | `97` | `93` | `+4` | Both strong, but this variant pushes further |
-| `DataExtract-15` | `81` | `83` | `-2` | Original edges ahead on extraction tasks |
-| `InstructFollow-15` | `94` | `77` | `+17` | Notably stronger instruction-following and stability |
-| `ReasonMath-15` | `79` | `79` | `0` | Essentially tied |
-| `StructOutput-15` | `88` | `77` | `+11` | Clear improvement in structured output |
-| `6-pack Total` | `86.5` | `78.2` | `+8.3` | Overall lead across the board |
-Two key takeaways:
-- Compared to the original `Qwen/Qwen3.6-27B q4_k_m`, this variant's `BenchLocal 6-pack` total is `8.3` points higher
-- It has proven itself capable of consistently ranking in the top tier across the full benchmark suite
-This means the advantage isn't just about one metric spiking — the overall score is notably elevated, and strengths are more evenly distributed.
-### Extended Eval Snapshot
-Current extended eval results for this `q4_k_m` build:
-| Eval | `Opus-DeepSeek-Distilled-Q4M` | `Qwen/Qwen3.6-27B q4_k_m` | Notes |
-| --- | --- | --- | --- |
-| `GPQA-Diamond-198` | `166 / 198 = 83.84%` | `146 / 198 = 73.7%` | Lead of `10.14` percentage points; all 198 questions locally graded |
-| `MMLU-500 (5-shot)` | `91.80%` (completed) | `91.6%` | Approximately tied |
-## Capability Profile
-### Core Strengths
-- **Strong scientific reasoning**
-  - GPQA consistently above `83%+`, proving it's not just an engineering-bench specialist.
-- **Broad knowledge coverage**
-  - MMLU-500 (5-shot) at `91.80%`, demonstrating competence across a wide range of standard knowledge tasks.
-- **Strong tool use and exam-style performance**
-  - Among the highest tiers in the `BenchLocal 6-pack` historical leaderboard.
-- **Reliable long-form task closure**
-  - Full GPQA set completed with scorable outputs under the current extended eval pipeline.
-## Recommended Use
-Scenarios best suited for this model:
-- General-purpose local agent
-- Complex multi-step task decomposition and execution
-- Engineering tasks requiring stable tool use and long-chain closure
-- Challenging knowledge Q&A and complex reasoning
-- General workflows needing structured thinking, action planning, and result delivery
-## Best Practices
-Deployment-wise, this version doesn't just "run" — with the right configuration it delivers an experience close to daily API usage.
-- Single `A100 40GB` at `q4_k_m + mtp=3` achieves approximately `60 tokens/s`
-- Single `RTX PRO 6000` at `q4_k_m + mtp=3` achieves approximately `100 tokens/s`
-- By local LLM deployment standards, this speed is smooth and perceptually close to API-level responsiveness
-- Recommended stack: **OpenCode + LM Studio** with the model loaded at `q4_k_m + mtp=3`. For coding tasks, we recommend `temperature = 0.6, top_p = 0.95` — this configuration delivers the best coding results in real-world usage
-- For long-running local deployments, it offers both scores and real interactive speed
-- For scenarios requiring high-intensity reasoning, agent usability, and day-to-day fluency, this configuration is a compelling practical choice
-## Known Limitations
-### Evaluation Methodology Limitations
-- **Opus-as-judge biases**: The "outperforms GLM5.2" comparisons in this report use Opus 4.8 as the judge. Opus-as-judge is known to exhibit preference for certain model styles — findings do not represent absolute rankings. Readers should treat these as signals, not definitive evaluations.
-- **MMLU-500 final value `91.80%`**: This is a single-run 5-shot result. Different shot selection strategies and prompt templates may cause score variation. Ensure evaluation methodology consistency when making cross-model comparisons.
-### Known Capability Boundaries
-- **DataExtract slightly below baseline**: This variant scores `81` on `BenchLocal DataExtract-15`, below the original `Qwen3.6-27B`'s `83`. The distillation process may have introduced some overwriting or style shift on pure extraction tasks. Caution is advised when extraction precision is critical.
-- **Closed-source teacher models**: The distillation process uses closed-source models (Claude Opus, DeepSeek) as teachers. Risks include inheriting the teacher's knowledge boundaries and biases, as well as potential terms-of-service compliance issues. Deployers should assess these risks for themselves.
-- **Model scale constraints**: A 27B-parameter model may still encounter capacity bottlenecks on extremely complex long-chain reasoning. Layered verification or backtracking strategies may be needed as compensation.
-### Deployment Considerations
-- **MTP=3 trade-off**: MTP (Multi-Token Prediction) at `q4_k_m` significantly boosts throughput (A100 ~60 tok/s, RTX PRO 6000 ~100 tok/s) but adds VRAM overhead. On memory-constrained hardware (e.g., under 24GB), consider disabling or reducing MTP depth.
-- **Quantization precision choices**: This report is based on `q4_k_m`. Higher-precision `q5_k_m` requires more VRAM but may yield accuracy gains. No systematic accuracy-vs-speed comparison across quantization levels has been performed. Choose based on your VRAM budget and accuracy requirements.
-- **imatrix notes**: This build uses the importance matrix (imatrix) quantization scheme, distinct from standard k-quant. Imatrix computes per-layer weight "importance" from a calibration dataset, better preserving critical parameter precision at low bit widths.
-## Bottom Line
-`Opus-DeepSeek-Distilled-Q4M` is a highly competitive, high-scoring model.
-Its most important value isn't just strong `BenchLocal` scores — it's that it delivers on harder, broader benchmarks like `GPQA-Diamond-198` and `MMLU-500 (5-shot)`.
-Even more compelling, its improvements go beyond numbers on an eval table. After extended real-world local use, its performance on engineering tasks like `BugFind` makes it clear: "this isn't the kind of model that only looks good on paper." When a `Qwen3.6-27B`-based local model can match `GLM5.2` in certain real-world scenarios, it has crossed from "usable" into "genuinely valuable candidate" territory.
-For anyone seeking higher benchmark scores, stronger scientific reasoning, and a higher ceiling on difficult problems from a local model, `Opus-DeepSeek-Distilled-Q4M` has proven itself worthy of a permanent spot on the evaluation and retention list.
-## Acknowledgements
-This release would not have been possible without the inspiration and support of many open-source teams, methodologies, and prior work.
-- Thanks to the `Qwen` team for providing an excellent foundational model that made subsequent agent design, distillation enhancements, and local deployment optimization possible.
-- Thanks to the `Unsloth` team for providing critical engineering support for efficient training, fine-tuning, and experimental iteration.
-- We also want to note that many of the observations and methodological judgments here are not original — they build on existing community exploration and extensive practical experience.
-- Special thanks to `Merkyor`, who was the first to clearly identify and validate the `ReAct` approach for reducing infinite loops in agent scenarios — this directly shaped the design direction of this release.
-It is because of this work that came before that we can continue pushing forward, arriving at today's more stable, more practical, and more complete agent — and helping us get closer to the era of local agent AI.

       - task:
           type: text-generation
         dataset:
           type: benchlocal
+          name: BenchLocal 6-pack
         metrics:
           - type: benchlocal-score
             value: 86.5
       - task:
           type: question-answering
         dataset:
           type: gpqa
+          name: GPQA-Diamond-198
         metrics:
           - type: accuracy
             value: 83.84
       - task:
           type: question-answering
         dataset:
           type: mmlu
+          name: MMLU-500 (5-shot)
         metrics:
           - type: accuracy
             value: 91.80
             name: Accuracy
 ---
+# 🧠 Opus-DeepSeek-Distilled-Q4M
+> A distilled Qwen3.6-27B GGUF optimized for local agentic reasoning, tool use, and long-chain task execution.
+[![BenchLocal](https://img.shields.io/badge/BenchLocal-86.5-8A2BE2)](https://github.com/brianyin/BenchLocal)
+[![GPQA](https://img.shields.io/badge/GPQA--Diamond--198-83.84%25-blue)](https://github.com/brianyin/BenchLocal)
+[![MMLU](https://img.shields.io/badge/MMLU--500-91.80%25-green)](https://github.com/brianyin/BenchLocal)
+[![License](https://img.shields.io/badge/License-Apache--2.0-orange)](https://www.apache.org/licenses/LICENSE-2.0)
+[![GGUF](https://img.shields.io/badge/Format-GGUF-ff69b4)](https://github.com/ggml-org/llama.cpp)
+---
+## 📢 Highlights
+| Area | Score | vs Qwen3.6-27B q4_k_m |
+|------|-------|----------------------|
+| **BenchLocal 6-pack** 🏆 | **86.5** | **+8.3** |
+| **GPQA-Diamond-198** 🔬 | **83.84%** | +10.14% |
+| **BugFind-15** 🐛 | **80** | **+20** |
+| **ToolCall-15** 🔧 | **97** | +4 |
+| **InstructFollow-15** 📋 | **94** | **+17** |
+| **StructOutput-15** 📊 | **88** | **+11** |
+| **MMLU-500 (5-shot)** 📚 | **91.80%** | ~tied (+0.2%) |
+| **DataExtract-15** 📄 | 81 | -2 |
+> 🚀 **Output speed**: `~60 tok/s` on A100 40GB · `~100 tok/s` on RTX PRO 6000 (q4_k_m + mtp=3)
+---
+## 🔥 Why This Model?
+The original `Qwen3.6-27B` has solid foundational capabilities, but its **agent behavior falls short** — prone to infinite loops when thinking, lacks structured agent design, and has room to improve in math reasoning.
+This variant tackles all three through targeted distillation:
+1. ✅ **Infinite loops → Eliminated.** ReAct-style reasoning-action orchestration fixes the root cause.
+2. ✅ **Agent behavior → Structured.** Distilled Claude Opus's systematic thinking and organization.
+3. ✅ **Math reasoning → Strengthened.** Absorbed capabilities from strong math/logic models.
+The result is a local agent that doesn't just score high on benchmarks — it *works* reliably in real engineering tasks like BugFind, where it now competes with GLM5.2.
+> ⚠️ **Note**: Side-by-side GLM5.2 comparisons were evaluated using Opus 4.8 as a judge. Opus-as-judge has inherent biases — results are indicative, not definitive.
+---
+## 🎯 Design Philosophy
+**Core insight**: `Qwen3.6-27B` doesn't lack capability — it lacks *good agent behavior*. That makes it worth iterating on.
+We followed the **[ReAct](https://arxiv.org/abs/2210.03629)** paradigm (Yao et al., ICLR 2023) — unifying reasoning and action into an alternating, constrained, executable loop — rather than just making the model "think longer" or "call tools better."
+| Teacher Model | Capability Distilled |
+|---------------|---------------------|
+| **Claude Opus** 🎯 | Systematic thinking, structured organization, concise reasoning |
+| **DeepSeek** 🧭 | Stable agent behavior, tool orchestration, task closure |
+| **Math/Logic models** ➗ | Mathematical reasoning, logical deduction |
+---
+## 📊 Performance
+### BenchLocal 6-pack
+| Pack | `q4_k_m` (ours) | `Qwen/Qwen3.6-27B q4_k_m` | Delta |
+|------|:-:|:-:|:-:|
+| **BugFind-15** 🐛 | **80** | 60 | **+20** |
+| **ToolCall-15** 🔧 | **97** | 93 | +4 |
+| **DataExtract-15** 📄 | 81 | **83** | -2 |
+| **InstructFollow-15** 📋 | **94** | 77 | **+17** |
+| **ReasonMath-15** ➗ | 79 | 79 | 0 |
+| **StructOutput-15** 📊 | **88** | 77 | **+11** |
+| **Total** 🏆 | **86.5** | 78.2 | **+8.3** |
+### Extended Evals
+| Benchmark | Ours | Baseline | Notes |
+|-----------|:----:|:--------:|-------|
+| **GPQA-Diamond-198** 🔬 | **83.84%** | 73.7% | +10.14%, all 198 graded locally |
+| **MMLU-500 (5-shot)** 📚 | **91.80%** | 91.6% | Approximately tied |
+---
+## 🛠️ Usage
+### Recommended Stack
+```
+🧩 OpenCode + LM Studio
+📐 Temperature: 0.6 · Top-p: 0.95
+⚡ q4_k_m + mtp=3
+```
+### Quick Start (llama.cpp)
+```bash
+# Download the GGUF
+huggingface-cli download your-org/Opus-DeepSeek-Distilled-Q4M \
+  opus-deepseek-distilled-q4m-q4_k_m.gguf --local-dir ./models
+# Run with llama.cpp
+./llama-cli -m ./models/opus-deepseek-distilled-q4m-q4_k_m.gguf \
+  --temp 0.6 --top-p 0.95 \
+  -p "Your prompt here"
+```
+### Via LM Studio
+1. Load the GGUF file in LM Studio
+2. Set backend to llama.cpp
+3. Enable MTP (set depth=3) under inference options
+4. Set `temperature = 0.6`, `top_p = 0.95`
+5. Start the local API server
+6. Connect via OpenCode or any OpenAI-compatible client
+> 💡 **Pro tip**: For coding tasks, the `temp 0.6 / top_p 0.95` combo delivers the best balance of creativity and correctness.
+---
+## ⚠️ Known Limitations
+### Evaluation Methodology
+- **Opus-as-judge biases**: GLM5.2 comparisons are judge-evaluated, not absolute rankings
+- **MMLU**: Single-run 5-shot result; variations in shot selection may cause fluctuation
+### Capability Boundaries
+- **DataExtract**: Scores 81 vs 83 baseline — extraction tasks may have slight regression from distillation
+- **Closed-source teachers**: Risks include inherited biases and TOS compliance — assess for your use case
+- **27B scale ceiling**: May still hit capacity limits on extremely complex long-chain reasoning
+### Deployment Notes
+- **MTP=3**: Boosts throughput but adds VRAM overhead — disable or reduce on <24GB hardware
+- **Quantization**: Only `q4_k_m` tested; `q5_k_m` may improve accuracy at higher VRAM cost
+- **Imatrix**: Uses importance-matrix quantization, not standard k-quant — better parameter preservation at low bit widths
+---
+## 📝 Citation
+```bibtex
+@misc{opus-deepseek-distilled-q4m,
+  title = {Opus-DeepSeek-Distilled-Q4M: A Distilled Agentic GGUF for Local Deployment},
+  author = {Yin, Brian and BenchLocal Contributors},
+  year = {2026},
+  url = {https://github.com/brianyin/BenchLocal}
+}
+```
+---
+## 🙏 Acknowledgements
+- **[Qwen](https://github.com/QwenLM/Qwen)** team — excellent foundational model
+- **[Unsloth](https://github.com/unslothai/unsloth)** — efficient training infrastructure
+- **[Merkyor](https://github.com/Merkyor)** — identified ReAct as the key to solving agent infinite loops
+- **Community** — built on existing open-source exploration and practical experience
+> *It is because of this work that came before that we can continue pushing forward, arriving at today's more stable, more practical, and more complete agent — and helping us get closer to the era of local agent AI.*