osmQwopus-3.6-27B-v2.1-uncensored-mxfp4

✅ VISION PRESERVED. Unlike most 4-bit quantization pipelines that degrade or strip the vision tower, this release keeps the full Qwen3.6-VL ViT and all projector weights in fp16. Use it as a real multimodal model at half the size of the 8-bit build.

MLX MXFP4 (OCP Microscaling FP4) quantization of a ZeroFuse-abliterated Qwopus 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from 91/100 → 4/100 with KL drift of just 0.0176 (measured at BF16). Quantization adds only +0.87% perplexity with 92.4% greedy-token agreement vs the BF16 reference. By the osmAPI research team and TERV.Pro student research team. Quantized with mlx-mtp — a vision-preserving MXFP4 quantizer.

Fastest absolute throughput of any osmQwopus MLX build — 27.7 tok/s vanilla on M4 Max. Apple Silicon is memory-bandwidth-bound: halving the model fits more weights in fast SRAM, which matters more than the precision drop.

---

⚡ TL;DR

Property	Value
Disk size	~14 GB
Effective BPW	4.25 (4-bit E2M1 data + E8M0 scale per group of 32)
Scheme	OCP Microscaling FP4 (MXFP4) — mode=mxfp4, group_size=32
Vision tower	✅ fp16 — all 333 ViT + projector weights at full precision
SSM params	✅ fp16 — a_log, dt_bias, conv1d kept at full precision
Refusal rate (ZeroFuse, n=100)	4/100 (vs vanilla Qwopus 91/100)
KL divergence vs BF16 (quant)	0.034 nats mean · JSD 0.009 (essentially identical)
Top-1 greedy agreement vs BF16	92.4% — same argmax token at 9 in 10 positions
PPL increase vs BF16	+0.87% — negligible quality loss
Throughput (M4 Max, vanilla)	27.7 tok/s — fastest of all osmQwopus MLX builds
Throughput + DFlash (b=8)	29.3 tok/s (1.06×)
Recommended RAM	20 GB+ Apple Silicon — fits M2 Pro / M3 Pro and above
Best for	Fastest local inference; 16–32 GB Macs; vision + text
Released by	osmAPI · TERV.Pro

---

🎚️ All osmQwopus variants

The full osmQwopus family from osmAPI — same ZeroFuse-abliterated weights (refusal 4/100, KL 0.0176), different quant schemes for different runtimes.

Quant	Format	BPW	Disk	Vision	Runtime	Link
8-bit	MLX	8.50	~27 GB	✅ native	mlx-vlm	…-8-bit-mlx
6-bit	MLX	6.66	~21 GB	✅ native	mlx-vlm	…-6-bit-mlx
MXFP4 (this repo)	MLX	4.25	~14 GB	✅ fp16 tower	mlx-mtp / mlx-vlm	— (you are here)
OptiQ 3.7bpw	MLX	~3.7	~14 GB	✅ ViT spliced	mlx-vlm	…-OptiQ-3.7bpw-mlx
Q8_0	GGUF	8.50	~28 GB	✅ via mmproj	llama.cpp	…-8-bit-GGUF
Q6_K	GGUF	~6.56	~22 GB	✅ via mmproj	llama.cpp	…-6-bit-GGUF
Q4_K_M	GGUF	~4.92	~16 GB	✅ via mmproj	llama.cpp	…-Q4_K_M-GGUF
TQ3_4S	GGUF	4.00 (~3.5 eff)	~14 GB	✅ via mmproj	llama.cpp-tq3	…-TQ3_4s-GGUF
TQ3_1S	GGUF	4.00 (~3.5 eff)	~14 GB	✅ via mmproj	llama.cpp-tq3	…-TQ3_1s-GGUF

👉 All variants share the same abliterated base weights — pick by your runtime (Apple Silicon → MLX; CUDA/CPU/cross-platform → GGUF) and your RAM budget. MXFP4 is the fastest MLX build on M4 Max; use 8-bit if you want the MTP head for native speculative decoding without a drafter.

---

🧬 Lineage

Qwen/Qwen3.6-27B                              (Qwen Team — base multimodal pretrain)
        │
        ▼
Jackrong/Qwopus3.6-27B-v2                     (Jackrong — Claude-Opus reasoning distill)
        │
        ▼
ZeroFuse abliteration (TPE-50)          (osmAPI · TERV.Pro)
   ├── 25 random startup trials
   ├── 2 community priors (coder3101, wangzhang)
   └── 23 TPE smart-sampling trials → best at trial 45
        │
        ▼
osmapi/osmQwopus-3.6-27B-V2-zerofuse           (BF16 full-precision checkpoint)
        │
        ▼
this repo — MLX MXFP4 quant                  (junainfinity/mlx-mtp · osmAPI · TERV.Pro)

Direct upstream links:

• 🏛️ Foundation: Qwen/Qwen3.6-27B
• 🎓 Claude-Opus distill: Jackrong/Qwopus3.6-27B-v2
• 🔓 Abliteration tool: ZeroFuse by osmAPI
• 🧮 Quantization tool: mlx-mtp — vision-preserving MXFP4 quantizer

---

📊 Abliteration Results

ZeroFuse measures refusals on mlabonne/harmful_behaviors (100 hard red-team prompts) and KL divergence on mlabonne/harmless_alpaca.

Stage	Refusals (n=100) ↓	KL divergence ↓
Vanilla Jackrong/Qwopus3.6-27B-v2	91 / 100	— (reference)
Community prior: coder3101 (T27)	4 / 100	0.0359
Community prior: wangzhang (T28)	30 / 100	0.0259
TPE best (T45) — BF16 source	4 / 100	0.0176
MXFP4 quant (this model)	4 / 100	0.034 nats vs BF16

→ 96% reduction in refusals with capability preserved at BF16. MXFP4 quantization adds only +0.87% perplexity on top — the abliteration and quantization quality loss are both negligible.

---

🧪 Method

Abliteration (inherited from the BF16 source) — TPE-50 Optuna search on ZeroFuse, M4 Max 128 GB. Full method in that card.

MXFP4 quantization (this repo):

1. Source — loaded osmapi/osmQwopus-3.6-27B-V2-abliterated-uncensored (51 GB BF16, 3 shards) into MLX on M4 Max.
2. Layer audit — mlx_mtp.mxfp4_quantize catalogued 1199 tensors: identified all LM linear projections (attention Q/K/V/O + MLP gate/up/down + lm_head) as MXFP4 candidates; flagged 196 vision-tower + projector tensors and 48 SSM-sensitive params (a_log, dt_bias, conv1d) for fp16 preservation.
3. Quantization — LM linears → MXFP4 (OCP MX E2M1, group_size=32) using mlx.core.quantize. Vision + SSM tensors written verbatim at fp16. 15 MTP head weights absent from source — quantizer set mtp_num_hidden_layers: 0 in output config to prevent strict-load failure.
4. Output — 3 safetensor shards, 14 GB total. Embedded mlx_mtp config block records format, vision_fp16=True, mtp_preserved=False.
5. Verification — text generation correct; vision captioning correct (image → caption matches BF16 output); DFlash block-diffusion speculative decoding loads and runs; KL divergence measured end-to-end (see Quality section below).

Total wall-clock: ~2 h on M4 Max 128 GB (dominated by disk I/O).

---

📊 Quantization quality vs BF16

Measured via teacher-forced prefill — 10 diverse prompts (code, math, reasoning, instruction-following), 244 token positions, full 248K-vocab distributions at every position.

Metric	Value	Interpretation
KL(bf16 ‖ mxfp4) mean	0.034 nats	Forward divergence from reference
JSD mean	0.009 nats	1.3% of [0, ln2] bound — essentially identical
Top-1 agreement	92.4%	Both pick same greedy token at 9 in 10 positions
PPL increase	+0.87%	Perplexity vs BF16 reference tokens
Code prompts (KL)	0.010–0.012	Lowest — code is deterministic

JSD < 0.01 is the standard "essentially identical distributions" threshold. At 0.009 we are below it. The 7.6% of positions where models disagree on the top-1 token are near-tie argmax flips from quantization noise — not quality regressions.

---

📦 Use it

Fastest inference via mlx-mtp (recommended — native DFlash support)

git clone https://github.com/jundot/omlx && pip install -e ./omlx
git clone https://github.com/junainfinity/mlx-mtp && pip install -e ./mlx-mtp

from mlx_mtp.engine import load_model, vanilla_generate

model, processor, config = load_model("osmapi/osmQwopus-3.6-27B-v2.1-uncensored-mxfp4")

result = vanilla_generate(
    model, processor, config,
    text="Write a Python function that implements merge sort:",
    max_tokens=512,
)
print(result["text"])
print(f"{result['tps']:.1f} tok/s")

With DFlash speculative decoding (block size 8 is optimal for mxfp4)

from mlx_mtp.engine import load_model
from mlx_mtp.dflash import load_dflash_drafter, dflash_generate

model, processor, config = load_model("osmapi/osmQwopus-3.6-27B-v2.1-uncensored-mxfp4")
drafter, _              = load_dflash_drafter("z-lab/Qwen3.6-27B-DFlash")

result = dflash_generate(
    model, processor, config, drafter,
    text="Explain the key differences between TCP and UDP:",
    max_tokens=512,
    draft_block_size=8,   # block 8 > 16 > 32 on mxfp4 (model is fast; keep overhead low)
)
print(result["text"])
print(f"{result['tps']:.1f} tok/s")

Inference via mlx-vlm

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("osmapi/osmQwopus-3.6-27B-v2.1-uncensored-mxfp4")
config = load_config("osmapi/osmQwopus-3.6-27B-v2.1-uncensored-mxfp4")

messages = [{"role": "user", "content": "Explain the difference between SSM and softmax attention in 3 sentences."}]
prompt = apply_chat_template(processor, config, messages)
print(generate(model, processor, prompt, max_tokens=400, verbose=True))

With an image

out = generate(
    model, processor,
    prompt=apply_chat_template(processor, config, [{"role": "user", "content": "Describe this picture."}], num_images=1),
    image=["./photo.jpg"],
    max_tokens=400, verbose=True,
)
print(out)

OpenAI-compatible HTTP via mlx-omni-server

pip install mlx-omni-server
mlx-omni-server  # serves on http://127.0.0.1:10240

curl -s http://127.0.0.1:10240/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "osmapi/osmQwopus-3.6-27B-v2.1-uncensored-mxfp4",
    "messages": [
      {"role": "system", "content": "Be brief and direct."},
      {"role": "user", "content": "Write a 3-line haiku about a heisenbug."}
    ],
    "max_tokens": 200
  }' | jq -r '.choices[0].message.content'

---

🧪 Quantization details

• Source weights: osmapi/osmQwopus-3.6-27B-V2-abliterated-uncensored — BF16, 3 shards, ~51 GB.
• Quantization scheme: OCP Microscaling FP4 (MXFP4) — 4-bit E2M1 mantissa (2 exponent bits, 1 mantissa bit) + shared E8M0 scale per group of 32 elements. Hardware-accelerated on Apple M-series via MLX's native MX support.
• Group size: 32.
• Effective bits/weight: 4 + 8/32 = 4.25 bpw (8-bit E8M0 scale shared across 32 weights).
• Vision tower: preserved verbatim in fp16 — 333 vision weights (ViT encoder, vision projector, image-token embeddings, deepstack indexes). Pixel/patch features degrade visibly at 4-bit; keeping them fp16 is free (vision = minority of total params).
• SSM-sensitive params: preserved in fp16 — 48 params (a_log, dt_bias, conv1d). Qwen3.5's hybrid-SSM recurrent state is numerically sensitive; fp16 costs nothing at scale.
• MTP head: disabled (mtp_num_hidden_layers: 0). The BF16 source had 0 MTP weights; quantizer detects this and clears the config to prevent strict-load failure. For native MTP speculative decoding, use the 8-bit build.
• Quantizer: mlx_mtp.mxfp4_quantize.

Architecture notes

Qwen 3.6 27B uses a hybrid attention stack — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The hybrid is fully supported by mlx-vlm 0.5.0+. For inference latency at MXFP4, expect 27–30 tok/s on M4 Max 128 GB at batch size 1 — approximately 2× faster than the 8-bit build because Apple Silicon is memory-bandwidth-bound and the smaller model fits more weights in fast SRAM per cycle.

---

⚠️ Behavior caveats

• Uncensored. Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
• Identity preserved. The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
• Heavy chain-of-thought. Qwopus inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like "Be brief and direct. Skip your reasoning.".

---

🙏 Credits

MXFP4 quantization

junainfinity/mlx-mtp — vision-preserving MXFP4 + oQ8 quantizer + native MTP / DFlash speculative decode engine for Apple Silicon VLMs.

Quantization & release

osmAPI research team — "OpenRouter of India" TERV.Pro student research team

Claude-Opus reasoning distill

Jackrong — author of Qwopus3.6-27B-v2

Foundation model

Qwen Team @ Alibaba Tongyi Lab — Qwen3.6-27B

Abliteration toolkit

ZeroFuse by osmAPI — Optuna-driven refusal-direction ablation with KL guardrails.

Community priors that seeded the TPE search

coder3101/Qwen3.5-27B-zerofuse · wangzhang/Qwen3.6-27B-abliterated

MLX stack

mlx-vlm · mlx (Apple)

---

📜 License

Apache-2.0, inherited from the foundation (Qwen3.6-27B) and the distill (Qwopus3.6-27B-v2) upstream.

---

Need a hosted endpoint, custom quant, or larger-scale inference? osmAPI — multi-provider LLM routing for the Indian developer ecosystem.

---

⚡ Faster decoding with DFlash (MLX, block-diffusion speculative)

This MLX build supports block-diffusion speculative decoding via DFlash — no requantization, no model changes. Because MXFP4 runs at ~27.7 tok/s baseline (already fast), DFlash gains are modest vs the 8-bit build; use block size 8 for the best result.

Apple M4 Max · 256 tok greedy · 2026-06-11
  MXFP4  vanilla          27.7 tok/s   1.00×
  MXFP4  + DFlash b=8     29.3 tok/s   1.06×  ← optimal
  MXFP4  + DFlash b=16    26.2 tok/s   0.95×
  MXFP4  + DFlash b=32    24.2 tok/s   0.87×

Why the modest speedup? DFlash's drafter overhead becomes a larger fraction of total time as baseline throughput rises. The 8-bit build sees ~3× gains because it starts slower; mxfp4 is already bandwidth-limited in a faster regime. If raw throughput is the priority, mxfp4 vanilla at 27.7 tok/s already beats oQ8 + DFlash (best = 23 tok/s).

Via mlx-mtp (recommended — correct block-size selection):

from mlx_mtp.engine import load_model
from mlx_mtp.dflash import load_dflash_drafter, dflash_generate

model, processor, config = load_model("osmapi/osmQwopus-3.6-27B-v2.1-uncensored-mxfp4")
drafter, _ = load_dflash_drafter("z-lab/Qwen3.6-27B-DFlash")

out = dflash_generate(model, processor, config, drafter,
                      text="Write a merge function for two sorted lists in Python.",
                      max_tokens=256, draft_block_size=8)
print(out["text"])

Via mlx-vlm CLI (standard interface):

python3 -m mlx_vlm generate \
  --model osmapi/osmQwopus-3.6-27B-v2.1-uncensored-mxfp4 \
  --draft-model z-lab/Qwen3.6-27B-DFlash --draft-kind dflash \
  --prompt "Write a merge function for two sorted lists in Python." --max-tokens 256

• Requires access to the gated drafter z-lab/Qwen3.6-27B-DFlash (one-click "Agree and access").
• Accelerates the text path only (vision encoding is unaffected).
• Adds ~3.9 GB for the drafter.
• Full benchmark write-up: mlx-mtp.