osmQwopus-3.6-27B-V2-abliterated-uncensored-8-bit-mlx

✅ VISION PRESERVED. Unlike most abliteration pipelines that strip the vision tower, this release keeps the full Qwen3.6-VL ViT (333 vision weights intact). Use it as a real multimodal model.

MLX 8-bit affine quantization of a ZeroFuse-abliterated Qwopus 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from 91/100 → 4/100 with KL drift of just 0.0176. By the osmAPI research team and TERV.Pro student research team.

---

⚡ TL;DR

Property	Value
Disk size	~27 GB
Effective BPW	8.5 (8 bits + 16-bit scale/bias per group of 64)
Scheme	MLX affine, group_size=64, mode=affine, bits=8
Refusal rate (ZeroFuse, n=100)	4/100 (vs vanilla Qwopus 91/100)
KL divergence vs vanilla	0.0176
Vision	✅ Multimodal (333 ViT weights preserved)
Recommended RAM	36 GB+ Apple Silicon (M-series Pro / Max / Ultra)
Best for	Highest-fidelity local inference; vision + text
Released by	osmAPI · TERV.Pro

---

🎚️ All osmQwopus variants

The full osmQwopus family from osmAPI — same ZeroFuse-abliterated weights (refusal 4/100, KL 0.0176), different quant schemes for different runtimes.

Quant	Format	BPW	Disk	Vision	Runtime	Link
8-bit (this repo)	MLX	8.50	~27 GB	✅ native	mlx-vlm	— (you are here)
6-bit	MLX	6.66	~21 GB	✅ native	mlx-vlm	…-6-bit-mlx
OptiQ 3.7bpw	MLX	~3.7	~14 GB	✅ ViT spliced	mlx-vlm	…-OptiQ-3.7bpw-mlx
Q8_0	GGUF	8.50	~28 GB	✅ via mmproj	llama.cpp	…-8-bit-GGUF
Q6_K	GGUF	~6.56	~22 GB	✅ via mmproj	llama.cpp	…-6-bit-GGUF
Q4_K_M	GGUF	~4.92	~16 GB	✅ via mmproj	llama.cpp	…-Q4_K_M-GGUF
TQ3_4S	GGUF	4.00 (~3.5 eff)	~14 GB	✅ via mmproj	llama.cpp-tq3	…-TQ3_4s-GGUF
TQ3_1S	GGUF	4.00 (~3.5 eff)	~14 GB	✅ via mmproj	llama.cpp-tq3	…-TQ3_1s-GGUF

👉 All variants share the same abliterated base weights — pick by your runtime (Apple Silicon → MLX; CUDA/CPU/cross-platform → GGUF) and your RAM budget.

---

🧬 Lineage

Qwen/Qwen3.6-27B                              (Qwen Team — base multimodal pretrain)
        │
        ▼
Jackrong/Qwopus3.6-27B-v2                     (Jackrong — Claude-Opus reasoning distill)
        │
        ▼
ZeroFuse abliteration (TPE-50)          (osmAPI · TERV.Pro)
   ├── 25 random startup trials
   ├── 2 community priors (coder3101, wangzhang)
   └── 23 TPE smart-sampling trials → best at trial 45
        │
        ▼
this repo — MLX 8-bit affine quant            (osmAPI · TERV.Pro)

Direct upstream links:

• 🏛️ Foundation: Qwen/Qwen3.6-27B
• 🎓 Claude-Opus distill: Jackrong/Qwopus3.6-27B-v2
• 🔓 Abliteration tool: ZeroFuse by osmAPI
• 🧮 Quantization tool: mlx-vlm

---

📊 Abliteration Results

ZeroFuse measures refusals on mlabonne/harmful_behaviors (100 hard red-team prompts) and KL divergence on mlabonne/harmless_alpaca.

Stage	Refusals (n=100) ↓	KL divergence ↓
Vanilla Jackrong/Qwopus3.6-27B-v2	91 / 100	— (reference)
Community prior: coder3101 (T27)	4 / 100	0.0359
Community prior: wangzhang (T28)	30 / 100	0.0259
TPE best (T45) — shipped here	4 / 100	0.0176
TPE second-best (T37)	5 / 100	0.0210

→ 96% reduction in refusals with capability preserved (KL ≈ 0.018, well below the 0.3 healing threshold). No SFT / LoRA healing was required.

---

🧪 Method (TPE-50 with community priors)

1. Smoke — 3-trial dry run on ZeroFuse / Python 3.11 / MPS to validate batch size 4 was feasible on M4 Max 128 GB.
2. Random search — 25 startup trials over ZeroFuse's full parameter space (direction_index, attn.o_proj.{max,min}_weight[_position,_distance], mlp.down_proj.{max,min}_weight[_position,_distance]). Best random-stage trial: T4 at direction_index=54.49, refusals 41/100, KL 0.0073 — capability-clean but still too refusing.
3. Community-prior enqueueing — published Qwen 27B ZeroFuse runs (coder3101 on Qwen 3.5, wangzhang on Qwen 3.6) consistently localize the refusal direction near layer 35–38 of 64. We injected both as study priors via study.enqueue_trial(skip_if_exists=True).
4. TPE smart-sampling — 23 Tree-structured Parzen Estimator trials refined around the community neighborhood. Trial 45 (direction_index=41.42) found the new Pareto front: 4/100 refusals at KL=0.0176.
5. Auto-save — best Pareto trial merged into base weights via ZeroFuse's LoRA-adapter merge path; saved as BF16 safetensors with vision weights preserved (333 weights).
6. MLX conversion — mlx_vlm.convert --hf-path ... -q --q-bits 8 --trust-remote-code produced this 8-bit variant. Vision weights verified post-conversion (333 still present).

Total wall-clock: ~13 h on M4 Max 128 GB.

In-place ZeroFuse patches used (env-var driven, no TTY required):

• ZEROFUSE_AUTO_SAVE — bypass interactive menu, auto-save Pareto-best.
• ZEROFUSE_AUTO_CONTINUE — non-interactive checkpoint resume keeping CLI settings.
• ZEROFUSE_ENQUEUE_PRIORS — load JSON priors before study.optimize.

---

📦 Use it

Inference via mlx-vlm (recommended — keeps vision)

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("osmapi/osmQwopus-3.6-27B-V2-abliterated-uncensored-8-bit-mlx")
config = load_config("osmapi/osmQwopus-3.6-27B-V2-abliterated-uncensored-8-bit-mlx")

messages = [{"role": "user", "content": "Explain the difference between SSM and softmax attention in 3 sentences."}]
prompt = apply_chat_template(processor, config, messages)
print(generate(model, processor, prompt, max_tokens=400, verbose=True))

With an image

out = generate(
    model, processor,
    prompt=apply_chat_template(processor, config, [{"role":"user","content":"Describe this picture."}], num_images=1),
    image=["./photo.jpg"],
    max_tokens=400, verbose=True,
)
print(out)

OpenAI-compatible HTTP via mlx-omni-server

pip install mlx-omni-server
mlx-omni-server  # serves on http://127.0.0.1:10240

curl -s http://127.0.0.1:10240/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "osmapi/osmQwopus-3.6-27B-V2-abliterated-uncensored-8-bit-mlx",
    "messages": [
      {"role": "system", "content": "Be brief and direct."},
      {"role": "user", "content": "Write a 3-line haiku about a heisenbug."}
    ],
    "max_tokens": 200
  }' | jq -r '.choices[0].message.content'

---

🧪 Quantization details

• Source weights: BF16 abliterated checkpoint (12 shards, ~50 GB) — ZeroFuse T45 merged into Jackrong/Qwopus3.6-27B-v2.
• Quantization scheme: MLX affine (per-group scale + bias, 8-bit signed integer weights).
• Group size: 64.
• Effective bits/weight: 8 + 32/64 = 8.5 bpw (32 bits of scale+bias overhead spread over 64 weights).
• Vision tower: preserved verbatim — 333 vision weights (ViT, vision projector, image-token embeddings, deepstack indexes) survive quantization at 8-bit.

Architecture notes

Qwen 3.6 27B uses a hybrid attention stack — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The hybrid is fully supported by mlx-vlm 0.5.0+. For inference latency at 8-bit, expect ~9-11 tok/s on M4 Max 128 GB at batch size 1.

---

⚠️ Behavior caveats

• Uncensored. Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
• Identity preserved. The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
• Heavy chain-of-thought. Qwopus inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like "Be brief and direct. Skip your reasoning.".

---

🙏 Credits

Quantization & release

osmAPI research team — "OpenRouter of India" TERV.Pro student research team

Claude-Opus reasoning distill

Jackrong — author of Qwopus3.6-27B-v2

Foundation model

Qwen Team @ Alibaba Tongyi Lab — Qwen3.6-27B

Abliteration toolkit

ZeroFuse by osmAPI — Optuna-driven refusal-direction ablation with KL guardrails.

Community priors that seeded our TPE search

coder3101/Qwen3.5-27B-zerofuse · wangzhang/Qwen3.6-27B-abliterated

MLX quantization stack

mlx-vlm · mlx (Apple)

---

📜 License

Apache-2.0, inherited from the foundation (Qwen3.6-27B) and the distill (Qwopus3.6-27B-v2) upstream.

---

Need a hosted endpoint, custom quant, or larger-scale inference? osmAPI — multi-provider LLM routing for the Indian developer ecosystem.

⚡ 3.3–3.7× faster decoding with DFlash (lossless, MLX)

This MLX build supports lossless block-diffusion speculative decoding via DFlash in mlx_vlm — no requantization, no model changes. On an Apple M4 Max we measured 3.38× (8-bit) and 3.67× (bf16) decode speedups with byte-identical output; other MLX quants of this model should see a similar ~3×.

python3 -m mlx_vlm generate \
  --model osmapi/osmQwopus-3.6-27B-V2-abliterated-uncensored-8-bit-mlx \
  --draft-model z-lab/Qwen3.6-27B-DFlash --draft-kind dflash \
  --prompt "Write a merge function for two sorted lists in Python." --max-tokens 256

• Requires mlx_vlm ≥ 0.5.0 and access to the gated drafter z-lab/Qwen3.6-27B-DFlash (one-click "Agree and access").
• Accelerates the text path only (vision is unaffected); adds ~3.9 GB for the drafter.
• Acceptance ≈ 8.95 tokens/round (block size 16); the target runs ~10× fewer forward passes.
• Full write-up & benchmarks: [https://huggingface.co/blog/junafinity/block-diffusion-on-apple-silicon-with-3-7x-speedup] · see also DFLASH_SPECULATIVE_DECODING.md.