---
license: apache-2.0
language:
- en
- zh
- multilingual
tags:
- text-generation
- image-text-to-text
- mlx
- mlx-vlm
- safetensors
- qwen
- qwen3
- qwen3.5
- qwen3.6
- claude-opus-distill
- reasoning
- vision
- multimodal
- abliterated
- refusal-ablated
- uncensored
- apple-silicon
- mxfp4
- ocp-mx
- speculative-decoding
- dflash
- conversational
base_model:
- Jackrong/Qwopus3.6-27B-v2
- Qwen/Qwen3.6-27B
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
---

# Qwen3.6-27B-v2.1-uncensored-mxfp4

> ✅ **VISION PRESERVED.** Unlike most 4-bit quantization pipelines that degrade or strip the vision tower, this release keeps the full Qwen3.6-VL ViT and all projector weights in **fp16**. Use it as a real multimodal model at half the size of the 8-bit build.

**MLX MXFP4 (OCP Microscaling FP4) quantization** of a ZeroFuse-abliterated Qwen 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from **91/100 → 4/100** with KL drift of just **0.0176** (measured at BF16). Quantization adds only **+0.87% perplexity** with **92.4% greedy-token agreement** vs the BF16 reference. By the Lemura Labs research team. Quantized with **[mlx-mtp](https://github.com/junainfinity/mlx-mtp)** — a vision-preserving MXFP4 quantizer.

> Fastest absolute throughput of any Qwen3.6-27B MLX build — **27.7 tok/s vanilla** on M4 Max. Apple Silicon is memory-bandwidth-bound: halving the model fits more weights in fast SRAM, which matters more than the precision drop.

---

## ⚡ TL;DR

| Property | Value |
|----------|-------|
| **Disk size** | ~14 GB |
| **Effective BPW** | 4.25 (4-bit E2M1 data + E8M0 scale per group of 32) |
| **Scheme** | OCP Microscaling FP4 (MXFP4) — `mode=mxfp4`, `group_size=32` |
| **Vision tower** | ✅ fp16 — all 333 ViT + projector weights at full precision |
| **SSM params** | ✅ fp16 — `a_log`, `dt_bias`, `conv1d` kept at full precision |
| **Refusal rate (ZeroFuse, n=100)** | **4/100** (vs vanilla Qwen 3.6 **91/100**) |
| **KL divergence vs BF16 (quant)** | **0.034 nats** mean · **JSD 0.009** (essentially identical) |
| **Top-1 greedy agreement vs BF16** | **92.4%** — same argmax token at 9 in 10 positions |
| **PPL increase vs BF16** | **+0.87%** — negligible quality loss |
| **Throughput (M4 Max, vanilla)** | **27.7 tok/s** — fastest of all Qwen3.6-27B MLX builds |
| **Throughput + DFlash (b=8)** | **29.3 tok/s** (1.06×) |
| **Recommended RAM** | 20 GB+ Apple Silicon — fits M2 Pro / M3 Pro and above |
| **Best for** | Fastest local inference; 16–32 GB Macs; vision + text |
| **Released by** | Lemura Labs |

---

## 🎚️ All Qwen3.6-27B variants

The full Qwen3.6-27B family from Lemura Labs — same ZeroFuse-abliterated weights (refusal 4/100, KL 0.0176), different quant schemes for different runtimes.

| Quant | Format | BPW | Disk | Vision | Runtime | Link |
|---|---|---|---|---|---|---|
| 8-bit | MLX | 8.50 | ~27 GB | ✅ native | mlx-vlm | [`…-8-bit-mlx`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-8-bit-mlx) |
| 6-bit | MLX | 6.66 | ~21 GB | ✅ native | mlx-vlm | [`…-6-bit-mlx`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-6-bit-mlx) |
| **MXFP4** **(this repo)** | MLX | **4.25** | **~14 GB** | ✅ **fp16 tower** | mlx-mtp / mlx-vlm | — *(you are here)* |
| OptiQ 3.7bpw | MLX | ~3.7 | ~14 GB | ✅ ViT spliced | mlx-vlm | [`…-OptiQ-3.7bpw-mlx`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-OptiQ-3.7bpw-mlx) |
| Q8_0 | GGUF | 8.50 | ~28 GB | ✅ via mmproj | llama.cpp | [`…-8-bit-GGUF`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-8-bit-GGUF) |
| Q6_K | GGUF | ~6.56 | ~22 GB | ✅ via mmproj | llama.cpp | [`…-6-bit-GGUF`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-6-bit-GGUF) |
| Q4_K_M | GGUF | ~4.92 | ~16 GB | ✅ via mmproj | llama.cpp | [`…-Q4_K_M-GGUF`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-Q4_K_M-GGUF) |
| TQ3_4S | GGUF | 4.00 (~3.5 eff) | ~14 GB | ✅ via mmproj | [llama.cpp-tq3](https://github.com/turbo-tan/llama.cpp-tq3) | [`…-TQ3_4s-GGUF`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-TQ3_4s-GGUF) |
| TQ3_1S | GGUF | 4.00 (~3.5 eff) | ~14 GB | ✅ via mmproj | [llama.cpp-tq3](https://github.com/turbo-tan/llama.cpp-tq3) | [`…-TQ3_1s-GGUF`](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-TQ3_1s-GGUF) |

> 👉 All variants share the same abliterated base weights — pick by your runtime (Apple Silicon → MLX; CUDA/CPU/cross-platform → GGUF) and your RAM budget. MXFP4 is the fastest MLX build on M4 Max; use 8-bit if you want the MTP head for native speculative decoding without a drafter.

---

## 🧬 Lineage

```
Qwen/Qwen3.6-27B (Qwen Team — base multimodal pretrain)
 │
 ▼
Jackrong/Qwopus3.6-27B-v2 (Jackrong — Claude-Opus reasoning distill)
 │
 ▼
ZeroFuse abliteration (TPE-50) (Lemura Labs · TERV.Pro)
 ├── 25 random startup trials
 ├── 2 community priors (coder3101, wangzhang)
 └── 23 TPE smart-sampling trials → best at trial 45
 │
 ▼
lemuralabs/Qwen3.6-27B-V2-zerofuse (BF16 full-precision checkpoint)
 │
 ▼
this repo — MLX MXFP4 quant (junainfinity/mlx-mtp · Lemura Labs · TERV.Pro)
```

Direct upstream links:

* 🏛️ Foundation: [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)
* 🎓 Claude-Opus distill: [Jackrong/Qwopus3.6-27B-v2](https://huggingface.co/Jackrong/Qwopus3.6-27B-v2)
* 🔓 Abliteration tool: [ZeroFuse](https://github.com/junainfinity/ZeroFuse) by Lemura Labs
* 🧮 Quantization tool: [mlx-mtp](https://github.com/junainfinity/mlx-mtp) — vision-preserving MXFP4 quantizer

---

## 📊 Abliteration Results

ZeroFuse measures refusals on `mlabonne/harmful_behaviors` (100 hard red-team prompts) and KL divergence on `mlabonne/harmless_alpaca`.

| Stage | Refusals (n=100) ↓ | KL divergence ↓ |
|---|---|---|
| Vanilla Jackrong/Qwopus3.6-27B-v2 | **91 / 100** | — (reference) |
| Community prior: coder3101 (T27) | 4 / 100 | 0.0359 |
| Community prior: wangzhang (T28) | 30 / 100 | 0.0259 |
| **TPE best (T45) — BF16 source** | **4 / 100** | **0.0176** |
| **MXFP4 quant (this model)** | **4 / 100** | **0.034 nats vs BF16** |

→ **96% reduction in refusals** with capability preserved at BF16. MXFP4 quantization adds only **+0.87% perplexity** on top — the abliteration and quantization quality loss are both negligible.

---

## 🧪 Method

**Abliteration** (inherited from the [BF16 source](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored)) — TPE-50 Optuna search on ZeroFuse, M4 Max 128 GB. Full method in that card.

**MXFP4 quantization** (this repo):

1. **Source** — loaded `lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored` (51 GB BF16, 3 shards) into MLX on M4 Max.
2. **Layer audit** — `mlx_mtp.mxfp4_quantize` catalogued 1199 tensors: identified all LM linear projections (attention Q/K/V/O + MLP gate/up/down + lm_head) as MXFP4 candidates; flagged 196 vision-tower + projector tensors and 48 SSM-sensitive params (`a_log`, `dt_bias`, `conv1d`) for fp16 preservation.
3. **Quantization** — LM linears → MXFP4 (OCP MX E2M1, `group_size=32`) using `mlx.core.quantize`. Vision + SSM tensors written verbatim at fp16. 15 MTP head weights absent from source — quantizer set `mtp_num_hidden_layers: 0` in output config to prevent strict-load failure.
4. **Output** — 3 safetensor shards, 14 GB total. Embedded `mlx_mtp` config block records format, vision_fp16=True, mtp_preserved=False.
5. **Verification** — text generation correct; vision captioning correct (image → caption matches BF16 output); DFlash block-diffusion speculative decoding loads and runs; KL divergence measured end-to-end (see Quality section below).

Total wall-clock: ~2 h on M4 Max 128 GB (dominated by disk I/O).

---

## 📊 Quantization quality vs BF16

Measured via teacher-forced prefill — 10 diverse prompts (code, math, reasoning, instruction-following), 244 token positions, full 248K-vocab distributions at every position.

| Metric | Value | Interpretation |
|---|---|---|
| **KL(bf16 ‖ mxfp4) mean** | **0.034 nats** | Forward divergence from reference |
| **JSD mean** | **0.009 nats** | 1.3% of [0, ln2] bound — essentially identical |
| **Top-1 agreement** | **92.4%** | Both pick same greedy token at 9 in 10 positions |
| **PPL increase** | **+0.87%** | Perplexity vs BF16 reference tokens |
| Code prompts (KL) | 0.010–0.012 | Lowest — code is deterministic |

JSD < 0.01 is the standard "essentially identical distributions" threshold. At 0.009 we are below it. The 7.6% of positions where models disagree on the top-1 token are near-tie argmax flips from quantization noise — not quality regressions.

---

## 📦 Use it

### Fastest inference via `mlx-mtp` (recommended — native DFlash support)

```bash
git clone https://github.com/jundot/omlx && pip install -e ./omlx
git clone https://github.com/junainfinity/mlx-mtp && pip install -e ./mlx-mtp
```

```python
from mlx_mtp.engine import load_model, vanilla_generate

model, processor, config = load_model("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")

result = vanilla_generate(
 model, processor, config,
 text="Write a Python function that implements merge sort:",
 max_tokens=512,
)
print(result["text"])
print(f"{result['tps']:.1f} tok/s")
```

### With DFlash speculative decoding (block size 8 is optimal for mxfp4)

```python
from mlx_mtp.engine import load_model
from mlx_mtp.dflash import load_dflash_drafter, dflash_generate

model, processor, config = load_model("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")
drafter, _ = load_dflash_drafter("z-lab/Qwen3.6-27B-DFlash")

result = dflash_generate(
 model, processor, config, drafter,
 text="Explain the key differences between TCP and UDP:",
 max_tokens=512,
 draft_block_size=8, # block 8 > 16 > 32 on mxfp4 (model is fast; keep overhead low)
)
print(result["text"])
print(f"{result['tps']:.1f} tok/s")
```

### Inference via `mlx-vlm`

```bash
pip install mlx-vlm
```

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")
config = load_config("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")

messages = [{"role": "user", "content": "Explain the difference between SSM and softmax attention in 3 sentences."}]
prompt = apply_chat_template(processor, config, messages)
print(generate(model, processor, prompt, max_tokens=400, verbose=True))
```

### With an image

```python
out = generate(
 model, processor,
 prompt=apply_chat_template(processor, config, [{"role": "user", "content": "Describe this picture."}], num_images=1),
 image=["./photo.jpg"],
 max_tokens=400, verbose=True,
)
print(out)
```

### OpenAI-compatible HTTP via `mlx-omni-server`

```bash
pip install mlx-omni-server
mlx-omni-server # serves on http://127.0.0.1:10240
```

```bash
curl -s http://127.0.0.1:10240/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4",
 "messages": [
 {"role": "system", "content": "Be brief and direct."},
 {"role": "user", "content": "Write a 3-line haiku about a heisenbug."}
 ],
 "max_tokens": 200
 }' | jq -r '.choices[0].message.content'
```

---

## 🧪 Quantization details

* **Source weights**: `lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored` — BF16, 3 shards, ~51 GB.
* **Quantization scheme**: OCP Microscaling FP4 (MXFP4) — 4-bit E2M1 mantissa (2 exponent bits, 1 mantissa bit) + shared E8M0 scale per group of 32 elements. Hardware-accelerated on Apple M-series via MLX's native MX support.
* **Group size**: 32.
* **Effective bits/weight**: 4 + 8/32 = **4.25 bpw** (8-bit E8M0 scale shared across 32 weights).
* **Vision tower**: preserved verbatim in fp16 — 333 vision weights (ViT encoder, vision projector, image-token embeddings, deepstack indexes). Pixel/patch features degrade visibly at 4-bit; keeping them fp16 is free (vision = minority of total params).
* **SSM-sensitive params**: preserved in fp16 — 48 params (`a_log`, `dt_bias`, `conv1d`). Qwen3.5's hybrid-SSM recurrent state is numerically sensitive; fp16 costs nothing at scale.
* **MTP head**: disabled (`mtp_num_hidden_layers: 0`). The BF16 source had 0 MTP weights; quantizer detects this and clears the config to prevent strict-load failure. For native MTP speculative decoding, use the [8-bit build](https://huggingface.co/lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-8-bit-mlx).
* **Quantizer**: [`mlx_mtp.mxfp4_quantize`](https://github.com/junainfinity/mlx-mtp/blob/main/mlx_mtp/mxfp4_quantize.py).

### Architecture notes

Qwen 3.6 27B uses a **hybrid attention stack** — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The hybrid is fully supported by `mlx-vlm` 0.5.0+. For inference latency at MXFP4, expect **27–30 tok/s** on M4 Max 128 GB at batch size 1 — approximately 2× faster than the 8-bit build because Apple Silicon is memory-bandwidth-bound and the smaller model fits more weights in fast SRAM per cycle.

---

## ⚠️ Behavior caveats

* **Uncensored.** Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
* **Identity preserved.** The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
* **Heavy chain-of-thought.** Qwen 3.6 inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like `"Be brief and direct. Skip your reasoning."`.

---

## 🙏 Credits

**MXFP4 quantization**

[junainfinity/mlx-mtp](https://github.com/junainfinity/mlx-mtp) — vision-preserving MXFP4 + oQ8 quantizer + native MTP / DFlash speculative decode engine for Apple Silicon VLMs.

**Quantization & release**

Lemura Labs

**Claude-Opus reasoning distill**

[Jackrong](https://huggingface.co/Jackrong) — author of [Qwen 3.6 27B-v2](https://huggingface.co/Jackrong/Qwopus3.6-27B-v2)

**Foundation model**

[Qwen Team @ Alibaba Tongyi Lab](https://huggingface.co/Qwen) — [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)

**Abliteration toolkit**

[ZeroFuse](https://github.com/junainfinity/ZeroFuse) by Lemura Labs — Optuna-driven refusal-direction ablation with KL guardrails.

**Community priors that seeded the TPE search**

[coder3101/Qwen3.5-27B-zerofuse](https://huggingface.co/coder3101/Qwen3.5-27B-zerofuse) · [wangzhang/Qwen3.6-27B-abliterated](https://huggingface.co/wangzhang/Qwen3.6-27B-abliterated)

**MLX stack**

[mlx-vlm](https://github.com/Blaizzy/mlx-vlm) · [mlx](https://github.com/ml-explore/mlx) (Apple)

---

## 📜 License

Apache-2.0, inherited from the foundation (Qwen3.6-27B) and the distill (Qwen 3.6 27B-v2) upstream.

---

Need a hosted endpoint, custom quant, or larger-scale inference? Lemura Labs — multi-provider LLM routing for the Indian developer ecosystem.

---

## ⚡ Faster decoding with DFlash (MLX, block-diffusion speculative)

This MLX build supports **block-diffusion speculative decoding** via **[DFlash](https://huggingface.co/z-lab/Qwen3.6-27B-DFlash)** — no requantization, no model changes. Because MXFP4 runs at ~27.7 tok/s baseline (already fast), DFlash gains are modest vs the 8-bit build; use block size 8 for the best result.

```
Apple M4 Max · 256 tok greedy · 2026-06-11
 MXFP4 vanilla 27.7 tok/s 1.00×
 MXFP4 + DFlash b=8 29.3 tok/s 1.06× ← optimal
 MXFP4 + DFlash b=16 26.2 tok/s 0.95×
 MXFP4 + DFlash b=32 24.2 tok/s 0.87×
```

**Why the modest speedup?** DFlash's drafter overhead becomes a larger fraction of total time as baseline throughput rises. The 8-bit build sees ~3× gains because it starts slower; mxfp4 is already bandwidth-limited in a faster regime. If raw throughput is the priority, mxfp4 vanilla at 27.7 tok/s already beats oQ8 + DFlash (best = 23 tok/s).

**Via mlx-mtp** (recommended — correct block-size selection):

```python
from mlx_mtp.engine import load_model
from mlx_mtp.dflash import load_dflash_drafter, dflash_generate

model, processor, config = load_model("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")
drafter, _ = load_dflash_drafter("z-lab/Qwen3.6-27B-DFlash")

out = dflash_generate(model, processor, config, drafter,
 text="Write a merge function for two sorted lists in Python.",
 max_tokens=256, draft_block_size=8)
print(out["text"])
```

**Via mlx-vlm CLI** (standard interface):

```bash
python3 -m mlx_vlm generate \
 --model lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4 \
 --draft-model z-lab/Qwen3.6-27B-DFlash --draft-kind dflash \
 --prompt "Write a merge function for two sorted lists in Python." --max-tokens 256
```

- Requires access to the gated drafter [`z-lab/Qwen3.6-27B-DFlash`](https://huggingface.co/z-lab/Qwen3.6-27B-DFlash) (one-click "Agree and access").
- Accelerates the **text** path only (vision encoding is unaffected).
- Adds **~3.9 GB** for the drafter.
- Full benchmark write-up: [mlx-mtp](https://github.com/junainfinity/mlx-mtp).