Hugging Face's logo

0xdfi
/
Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX

Text Generation
MLX
Safetensors
qwen3_5_moe
omlx
qwen3
qwen3.5
abliterated
quantized
reap
moe-pruning
tool-calling
long-context
conversational
6-bit
Model card Files
xet
 Community
How to use from
Pi
# Gated model: Login with a HF token with gated access permission
hf auth login
Start the MLX server 
# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "0xdfi/Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX"
Configure the model in Pi 
# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "0xdfi/Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX"
        }
      ]
    }
  }
}
Run Pi 
# Start Pi in your project directory:
pi
Quick Links

Model Access and Use Terms

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

By checking the required box below and selecting “Agree and Access Model,” you (for yourself and your heirs, successors, and assigns) represent that you are at least 18 years old (or the age of majority in your jurisdiction, if greater), have legal capacity and authority to enter this agreement, and have read and understood these terms. You acknowledge that the model weights, code, configurations, documentation, and related files in this repository (the “Artifact”) are an unofficial research derivative in a model lineage with intentionally reduced refusal or safety behavior, are not sponsored, endorsed, approved by, or affiliated with any upstream author or base-model provider, have not been safety-evaluated by the repository owner, and may generate or enable harmful, illegal, offensive, infringing, private, sensitive, inaccurate, or otherwise unsafe content or outcomes. You accept the Artifact “AS IS” and “WITH ALL FAULTS.” To the fullest extent permitted by applicable law, the repository owner, author, uploader, maintainers, contributors, licensors, upstream authors, and each of their respective affiliates, officers, employees, and agents (collectively, the “Released Parties”) disclaim all warranties and conditions, express, implied, statutory, or arising from course of dealing or usage of trade, including merchantability, fitness for a particular purpose, title, non-infringement, accuracy, reliability, and safety. You knowingly and voluntarily assume all risks arising from access to or any use, misuse, modification, fine-tuning, distribution, deployment, output, or reliance on the Artifact and, to the fullest extent permitted by applicable law, irrevocably release, waive, discharge, and covenant not to sue the Released Parties for any claim, loss, liability, damage, penalty, cost, or expense arising from or related to the Artifact, including claims based on ordinary negligence. The Released Parties will not be liable for indirect, incidental, special, consequential, exemplary, or punitive damages, and their total aggregate liability arising from or related to the Artifact will not exceed US $100. You are solely responsible for reviewing and validating outputs, implementing appropriate safeguards and human oversight, preventing access by minors, complying with all applicable laws, regulations, export and sanctions controls, and license terms, and all consequences of your access and use. You agree to defend, indemnify, and hold harmless the Released Parties from every third-party claim and resulting loss, liability, damage, penalty, cost, or expense, including reasonable attorneys’ fees, arising from or related to your access, use, misuse, modification, distribution, deployment, outputs, or violation of law or these terms. The Released Parties are intended third-party beneficiaries entitled to enforce this agreement. These access and risk-allocation terms supplement and do not alter the copyright permissions granted by any applicable license; if a conflict exists, the applicable license controls copyright permissions and these terms control repository access and risk allocation. This agreement is governed by New York law, excluding conflict-of-laws rules, and you consent to exclusive jurisdiction and venue in the state and federal courts located in New York County, New York. If any provision is held unenforceable, it will be modified and limited to the minimum extent necessary or severed, and the remainder will remain in full force and effect. Terms version 2026-07-23.

Log in or Sign Up to review the conditions and access this model content.

Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX

Research artifact. This model is derived from an abliterated parent model with reduced refusal behavior. It can produce harmful, unsafe, or policy-violating outputs. Do not deploy it without your own safety layer, access controls, and monitoring. This is an unofficial derivative release and is not an official Qwen or parent-model release.

Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX is a static-MoE-pruned and MLX-quantized derivative of wangzhang/Qwen3.5-122B-A10B-abliterated, built to preserve tool-calling behavior while reducing memory footprint enough to be practical on a 128 GB Apple Silicon machine.

The explicit target for this release was:

  • produce a Q6-class local MLX artifact of this model
  • preserve tool calling as well as possible through the prune + quant pipeline
  • make a 100k context window a realistic reliability target on a 128 GB machine

This release uses:

  • 20% static REAP expert pruning on the MoE layers
  • 205 / 256 routed experts kept per MoE layer
  • oQ6 MLX quantization
  • Tool-calling-oriented REAP calibration based on tryumanshow/ToolACE-Qwen-cleaned

What This Is

This is a deployment artifact aimed at local MLX inference, not a fine-tune. The goal was:

  • preserve tool-calling behavior as much as possible
  • reduce model size enough to fit comfortably on a 128 GB Mac
  • keep a path open for large-context workloads, with 100k context as the practical target

The final artifact is an MLX oQ6 model with 16 safetensor shards and about 76 GB on-disk size.

Lineage

This model should be understood as a derivative artifact of the abliterated parent, not as an official upstream model variant.

Build Summary

Pruning

  • Method: static REAP expert pruning with reap-mlx
  • Calibration focus: tool-calling preservation
  • Prune plan: 2448 / 12288 routed experts removed
  • Per-layer change: 256 -> 205 routed experts
  • Output BF16 checkpoint size: about 184 GB

Quantization

  • Method: oQ6 quantization with oMLX
  • Final size: about 76 GB
  • Total indexed weight size: 81,618,377,035 bytes
  • Quantization base: 6-bit, group_size=64, affine
  • Important layers such as routers remain protected by higher-bit overrides in the quantization config

Evaluation Snapshot

These are sanity checks and release notes, not a full benchmark paper.

Tool Calling

  • Direct smoke test: passed
  • The final quant emitted a valid XML tool call for a simple arithmetic function call instead of answering directly
  • ToolACE-style sampled smoke:
    • sample size: 8
    • tool-call outputs: 7
    • observed tool-call rate: 87.5%
    • artifact: eval-20260403-011600/toolace_toolcall_smoke.json

Standard Sanity Benchmark

  • HellaSwag slice: validation[:256]
  • acc: 62.5%
  • acc_norm: 73.046875%
  • evaluation time: 268.41 s
  • peak memory during that eval: 84.34 GB
  • artifact: eval-20260403-011600/hellaswag-256.raw.txt

Long Context Note

Large-context behavior was part of the target profile for this release:

  • this model successfully cold-loaded and began inference with 100k+ token prompts on a 128 GB Apple Silicon machine
  • a saved long-context coding suite also started at 105,484 prompt tokens

So the honest claim is:

  • tool calling survived the prune + quant pipeline
  • the model runs locally in MLX
  • 100k context was the target profile for this release on a 128 GB machine
  • 100k+ context appears reachable

Files

  • config.json
  • generation_config.json
  • chat_template.jinja
  • tokenizer.json
  • tokenizer_config.json
  • model.safetensors.index.json
  • 16 model shard files

Model config highlights:

  • model_type: qwen3_5_moe
  • num_experts: 205
  • num_experts_per_tok: 8

Tools and Reference Files

External tools, models, and datasets used or referenced for this artifact:

Included reference files in this repo:

Usage

MLX / mlx-lm 

from mlx_lm import load, generate

model, tokenizer = load("0xdfi/Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX")

messages = [
    {"role": "user", "content": "Reply with exactly the word OK."}
]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
)

text = generate(model, tokenizer, prompt=prompt, max_tokens=8, verbose=False)
print(text)

Tool Calling 

from mlx_lm import load, generate

model, tokenizer = load("0xdfi/Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX")

tools = [{
    "type": "function",
    "function": {
        "name": "add_numbers",
        "description": "Add two integers.",
        "parameters": {
            "type": "object",
            "properties": {
                "a": {"type": "integer"},
                "b": {"type": "integer"},
            },
            "required": ["a", "b"],
        },
    },
}]

messages = [{
    "role": "user",
    "content": "What is 7 plus 8? Use the available function.",
}]

prompt = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    add_generation_prompt=True,
    tokenize=False,
)

text = generate(model, tokenizer, prompt=prompt, max_tokens=128, verbose=False)
print(text)

Reproduction Notes

High-level local pipeline:

  1. Start from the abliterated 122B parent model in unquantized MLX/BF16 form.
  2. Convert ToolACE-Qwen-cleaned into REAP-ready rendered chat samples.
  3. Collect REAP telemetry with a windowed/layerwise workflow to stay within local memory limits.
  4. Build a 20% prune plan.
  5. Apply structural pruning to produce a 205-expert-per-layer BF16 checkpoint.
  6. Quantize the pruned checkpoint to oQ6.
  7. Run tool-calling and sanity evals on the final artifact.

One reproducibility caveat: the final oQ6 quantization completed with position-based sensitivity fallback because the intended proxy model path was not loadable in the local quantization path at the time of export.

Limitations

  • Tool-calling preservation was a design target, not a formal guarantee.
  • The ToolACE validation included only a small sampled smoke in this release note.
  • This is an MLX artifact, not a Transformers-format upload.

Credits

Disclaimer

This repository is provided for research and local inference experimentation. The parent model is abliterated and may answer requests that safer models refuse. The author of this derivative artifact is not responsible for misuse. Users are responsible for complying with applicable law, platform rules, and their own safety requirements.

Downloads last month
16
Safetensors
Model size
22B params
Tensor type
U8
·
U32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

6-bit

Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for 0xdfi/Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX

Base model

Qwen/Qwen3.5-122B-A10B
Finetuned
wangzhang/Qwen3.5-122B-A10B-abliterix
Quantized
(18)
this model