---
base_model: trjxter/Qwimi-3.6-27B-Coder-MTP-BF16
license: apache-2.0
language:
- en
library_name: transformers
tags:
- unsloth
- lora
- qwen3_5
- code
- sft
- coding
- tool-calling
- agentic
- mlx
- mlx-my-repo
pipeline_tag: text-generation
datasets:
- trjxter/Kimi-K2.7-CodingTraces-9000x
- trjxter/Kimi-K2.6-Technical-Reasoning-AddOn-3300x
- trjxter/Kimi-K2.6-Reasoning-3300x-WandB
- Jackrong/Kimi-K2.5-Reasoning-1M-Cleaned
- Jackrong/Claude-opus-4.7-TraceInversion-5000x
- minpeter/xlam-function-calling-60k-parsed
- SWE-Factory/DeepSWE-Agent-Kimi-K2-Trajectories-2.8K
- nebius/SWE-agent-trajectories
---

# Qwimi-3.6-27B-Coder-MTP-MLX-6bit

A **6-bit MLX quantization** of
`trjxter/Qwimi-3.6-27B-Coder-MTP-BF16`, a coding-focused SFT of Qwen 3.6 27B.

The model was trained in one mixed SFT run across coding, Qwen-native XML-style tool calling,
and agentic software-engineering trajectories. This repository is intended for local
inference on **Apple silicon** using `mlx-lm`.

> This is an MLX repository, not GGUF. It is not intended for `llama.cpp`, CUDA inference,
> or a GGUF-only Ollama workflow.

**Quant position:** balanced quality/size MLX release.  
**Primary tradeoff:** lower quantization loss than Q4 while remaining smaller than Q8.  
**Recommendation:** Recommended balanced release when unified memory permits.

---

## Contents

1. [Source and quantization](#1-source-and-quantization)
2. [Which MLX quant should I use?](#2-which-mlx-quant-should-i-use)
3. [Unified-memory guidance](#3-unified-memory-guidance)
4. [Installation](#4-installation)
5. [How to run](#5-how-to-run)
6. [Tool calling](#6-tool-calling)
7. [Training summary](#7-training-summary)
8. [Benchmarks](#8-benchmarks)
9. [Known limitations](#9-known-limitations)
10. [Verification and reproducibility](#10-verification-and-reproducibility)

---

## 1. Source and quantization

- **Merged source:** `trjxter/Qwimi-3.6-27B-Coder-MTP-BF16`
- **Base architecture:** Qwen 3.6 27B, approximately 27.8B parameters
- **Architecture family:** dense hybrid transformer with standard attention and GatedDeltaNet layers
- **Specialization:** text-only coding SFT; vision tower frozen
- **Format:** MLX model weights for Apple silicon
- **Nominal weight precision:** 6-bit
- **Expected conversion flag:** `--q-bits 6`
- **Validated SFT sequence length:** 16,384 tokens

The uploaded repository's `config.json` is the source of truth for quantization mode, group
size, and tensor metadata. This README intentionally does not invent a group size that may
differ from the actual conversion.

The source name includes `MTP`. Do not assume that conversion to MLX automatically preserves
or activates the same MTP runtime path used by a specific `llama.cpp` build.

---

## 2. Which MLX quant should I use?

Choose Q6 when you want the strongest balance between coding quality, memory use, and
context headroom.

Prefer Q4 when memory is the main constraint. Prefer Q8 when maximum fidelity is worth the
additional memory cost.

| MLX release | Relative memory use | Expected fidelity | Best fit |
|---|---:|---|---|
| Q4 | Lowest | Good, with the most quantization loss | Constrained memory and more context headroom |
| **Q6** | Medium | Strong balance | General recommendation |
| Q8 | Highest | Highest of the three | Maximum quality on larger-memory Macs |

Exact quality and memory behavior should be measured with the uploaded artifacts and your own
workload.

---

## 3. Unified-memory guidance

- **Practical target:** 48 GB unified memory or more.
- **32 GB Macs:** may be tight after model cache and other applications are included.
- **64 GB+ Macs:** provide comfortable room for longer contexts and agent tooling.

These are rough planning targets, not guarantees. Actual memory use depends on the exact file
size, group size, prompt length, generation length, cache configuration, macOS version, and
other applications using unified memory.

---

## 4. Installation

MLX requires Apple silicon.

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip mlx-lm
```

Qwen 3.6 / `qwen3_5` support is relatively new, so update `mlx-lm` before diagnosing model,
cache, server, or tool-parser problems.

---

## 5. How to run

### One-shot generation

```bash
mlx_lm.generate \
  --model trjxter/Qwimi-3.6-27B-Coder-MTP-MLX-6bit \
  --prompt "Write a Python function that reverses a linked list and include tests."
```

### Interactive chat

```bash
mlx_lm.chat --model trjxter/Qwimi-3.6-27B-Coder-MTP-MLX-6bit
```

### Python API

```python
from mlx_lm import generate, load

MODEL_ID = "trjxter/Qwimi-3.6-27B-Coder-MTP-MLX-6bit"
model, tokenizer = load(MODEL_ID)

messages = [
    {
        "role": "user",
        "content": "Implement an LRU cache in Python and explain the complexity.",
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=1024,
    verbose=True,
)

print(response)
```

### Local OpenAI-style server

```bash
mlx_lm.server \
  --model trjxter/Qwimi-3.6-27B-Coder-MTP-MLX-6bit \
  --host 127.0.0.1 \
  --port 8080
```

`mlx_lm.server` is most appropriate as a local development endpoint, not a hardened public
production service.

### Context guidance

The base architecture may expose a larger native context, but Qwimi was trained and validated
at **16,384 tokens**. Longer-context behavior should be tested separately.

---

## 6. Tool calling

Qwimi was trained on Qwen 3.6's native XML-style tool-call format rather than the older plain
JSON-object style.

```xml
<tool_call>
<function=search_web>
<parameter=query>latest MLX documentation</parameter>
</function>
</tool_call>
```

Recommendations:

1. Apply the repository's tokenizer chat template.
2. Test the exact `mlx-lm` version you plan to deploy.
3. Validate multi-turn tool calls, inserted tool results, and stop behavior.
4. Do not assume that working text generation guarantees correct server-side tool parsing.
5. Keep direct XML validation or a fallback parser for agent workflows.

---

## 7. Training summary

- **Base:** `unsloth/Qwen3.6-27B`
- **Model size:** approximately 27.8B parameters
- **Method:** 4-bit QLoRA during SFT, rank 64, alpha 64
- **Trainable parameters:** 466,911,232, approximately 1.68%
- **Training style:** one mixed SFT run, not three sequential stages
- **Total training rows:** 22,359
- **Training tokens:** approximately 72.5M
- **Maximum sequence length:** 16,384
- **Effective batch size:** 16
- **Epochs:** 1
- **Optimizer steps:** 1,398
- **Hardware:** 1× A100-SXM4-80GB
- **Vision tower:** frozen
- **Final eval loss:** coding 0.4761; tool-calling 0.0208; agentic 0.2892
- All three eval curves improved monotonically through the run with no observed overfitting signal.

| Domain | Train rows | Eval rows | Token share |
|---|---:|---:|---:|
| Coding | 16,083 | 1,778 | 82.5% |
| Tool-calling | 5,360 | 595 | 4.8% |
| Agentic | 917 | 102 | 12.7% |

`trjxter/Kimi-K2.7-CodingTraces-9000x` was used in full: 9,014 rows.

## 8. Benchmarks

Qwimi was compared against the base **Qwen 3.6 27B** model using equivalent
**Q6 GGUF builds with MTP enabled** and the same evaluation harness.

> **Important:** these results were produced by the Q6 GGUF / `llama.cpp` builds. They are
> included as evidence about the underlying Qwimi model, not as direct benchmarks of this MLX
> quant. MLX Q4, Q6, and Q8 should be benchmarked separately before making backend- or
> precision-specific claims.

### 8.1 Fully auto-scored 300-task custom benchmark

| Category | Tasks |
|---|---:|
| Coding | 150 |
| Tool-calling | 75 |
| Agentic workflows | 75 |
| **Total** | **300** |

| Model | Tasks | Pass rate | Average wall time | Total wall time |
|---|---:|---:|---:|---:|
| Base Qwen 3.6 27B Q6 MTP | 300 | 81.33% | 53.56 s | 267.79 min |
| **Qwimi 3.6 27B Coder Q6 MTP** | **300** | **82.67%** | **27.09 s** | **135.45 min** |

| Overall change | Result |
|---|---:|
| Absolute pass-rate gain | **+1.33 percentage points** |
| Relative pass-rate gain | **+1.64%** |
| Average wall-time reduction | **49.4%** |
| Effective task-level speedup | **~1.98×** |
| Total evaluation time saved | **132.34 minutes** |

#### Category results

| Category | Base pass rate | Qwimi pass rate | Accuracy delta | Base avg time | Qwimi avg time | Wall-time reduction |
|---|---:|---:|---:|---:|---:|---:|
| Custom coding | 81.33% | **85.33%** | **+4.00 pts** | 59.33 s | **21.16 s** | **64.3%** |
| Custom tool-calling | 88.00% | **89.33%** | **+1.33 pts** | 31.77 s | **27.13 s** | **14.6%** |
| Custom agentic | **74.67%** | 70.67% | **-4.00 pts** | 63.80 s | **38.92 s** | **39.0%** |

#### Pairwise outcomes

| Pairwise result | Tasks | Share |
|---|---:|---:|
| Qwimi passed and base failed | **23** | **7.67%** |
| Base passed and Qwimi failed | 19 | 6.33% |
| Both passed | 225 | 75.00% |
| Both failed | 33 | 11.00% |
| **Total** | **300** | **100%** |

Among the 42 decisive tasks where only one model passed, Qwimi won **23/42 = 54.8%**.

| Category | Qwimi-only wins | Base-only wins | Both pass | Both fail |
|---|---:|---:|---:|---:|
| Coding | **11** | 5 | 117 | 17 |
| Tool-calling | **4** | 3 | 63 | 5 |
| Agentic | 8 | **11** | 45 | 11 |
| **Total** | **23** | **19** | **225** | **33** |

**300-task verdict:** a slight overall quality win for Qwimi, driven by stronger coding and
structured tool-use performance, with a large reduction in end-to-end task time. Agentic
accuracy remains the clearest weakness.

### 8.2 Mixed 200-task benchmark

A separate 200-task run produced 400 raw answer rows. Categories with incomplete scorer or
export integration were omitted rather than counted as zero.

| Model | Auto-scored rows | Passed | Pass rate | Average wall time |
|---|---:|---:|---:|---:|
| Base Qwen 3.6 27B Q6 MTP | 151 | 46 | 30.46% | 92.86 s |
| **Qwimi 3.6 27B Coder Q6 MTP** | **166** | **51** | **30.72%** | **71.30 s** |

| Overall change | Result |
|---|---:|
| Absolute pass-rate gain | **+0.26 percentage points** |
| Relative pass-rate gain | **+0.85%** |
| Average wall-time reduction | **23.2%** |

On the 146 tasks with a valid paired auto-score:

| Pairwise result | Tasks | Share |
|---|---:|---:|
| **Qwimi better** | **7** | **4.79%** |
| Base better | 2 | 1.37% |
| Same result | 137 | 93.84% |
| **Total paired tasks** | **146** | **100%** |

Among the nine decisive paired tasks, Qwimi won **7/9 = 77.8%**.

| Category | Base pass rate | Qwimi pass rate | Delta |
|---|---:|---:|---:|
| HumanEval | 80.00% | **94.29%** | **+14.29 pts** |
| Personal XML tool tasks | 50.00% | **100.00%** | **+50.00 pts** |
| MBPP | **5.77%** | 1.92% | **-3.85 pts** |
| Tool-calling checks | 100% | 100% | Tie |
| Architecture checks | 100% | 100% | Tie |
| Debugging checks | 100% | 100% | Tie |

### 8.3 Interpreting the latency results

The measured advantage is **task-level wall time**, not proof of faster kernels or higher
per-token throughput.

```text
Task time ≈ prefill time + generated tokens × decode latency per token
```

SFT can reduce task time by producing shorter, more direct, better-formatted outputs and
reaching the stop condition sooner. A controlled comparison of prompt throughput, generation
throughput, and generated-token counts is required to claim a per-token speed advantage.

### 8.4 Benchmark limitations

- The published comparison used Q6 GGUF under `llama.cpp`, not MLX.
- MLX Q4, Q6, and Q8 were not benchmarked against one another.
- Wall time varies with Mac hardware, `mlx-lm` version, context length, cache behavior,
  generation length, and sampling settings.
- The custom suite is useful internal evidence, not a replacement for broad public benchmarks.
- Agentic auto-scoring cannot capture every aspect of repository-level engineering quality.


## 9. Known limitations

- **Text-only specialization:** the vision tower was frozen during SFT and was not improved.
- **Validated context:** training and evaluation used a maximum sequence length of 16,384
  tokens. Behavior beyond that length is unvalidated.
- **Agentic coverage:** agentic data was much smaller than the coding portion, and the custom
  benchmark showed a four-point agentic accuracy regression against the base model.
- **Quantization loss:** this MLX conversion adds quantization error on top of BF16.
- **Apple Silicon only:** MLX is designed for Apple silicon and is not a CUDA/Windows runtime.
- **Bleeding-edge architecture:** Qwen 3.6 uses the `qwen3_5` hybrid architecture with
  GatedDeltaNet layers. Use a current `mlx-lm` build and verify generation locally.
- **MTP caveat:** the name retains `MTP` from the source model, but this does not guarantee
  that MLX preserves or actively uses the same MTP acceleration path as the GGUF benchmark.
- **Tool-calling caveat:** XML-style tool output and server-side parsing can be version-sensitive.
  Validate multi-turn tools before relying on them in an agent workflow.
- **Server caveat:** `mlx_lm.server` is intended primarily as a local endpoint and is not a
  hardened production service.


---

## 10. Verification and reproducibility

Record the exact:

- `mlx-lm` version;
- source BF16 commit SHA;
- conversion command;
- quantization mode, bit width, and group size;
- output repository commit SHA;
- uploaded size and weight-shard checksums;
- smoke-test prompt and structural result.

Representative conversion command:

```bash
mlx_lm.convert \
  --hf-path trjxter/Qwimi-3.6-27B-Coder-MTP-BF16 \
  --mlx-path ./Qwimi-3.6-27B-Coder-MTP-MLX-6bit \
  -q \
  --q-bits 6
```

Add the actual `--q-group-size`, quantization mode, and upload arguments used during conversion
instead of assuming defaults.

### Suggested smoke tests

1. Plain coding generation.
2. Exact `<think>...</think>` formatting.
3. A single XML-style tool call.
4. A multi-turn tool call with a tool result.
5. A 12K-16K-token prompt to check memory and cache behavior.
6. Repeated generation to detect truncation or cache regressions.

---

## License

Apache 2.0, inherited from the base and merged source model.