Text Generation
Transformers
Safetensors
English
qwen3_5_moe_text
qwen
qwen3.5
Mixture of Experts
mixture-of-experts
abliterated
uncensored
refusal-removal
expert-ablation
mechanistic-interpretability
red-teaming
research
conversational
Instructions to use Carlosian/Qwen3.5-35B-A3B-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Carlosian/Qwen3.5-35B-A3B-Abliterated") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Carlosian/Qwen3.5-35B-A3B-Abliterated") model = AutoModelForCausalLM.from_pretrained("Carlosian/Qwen3.5-35B-A3B-Abliterated", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Carlosian/Qwen3.5-35B-A3B-Abliterated" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Carlosian/Qwen3.5-35B-A3B-Abliterated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Carlosian/Qwen3.5-35B-A3B-Abliterated
- SGLang
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Carlosian/Qwen3.5-35B-A3B-Abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Carlosian/Qwen3.5-35B-A3B-Abliterated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Carlosian/Qwen3.5-35B-A3B-Abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Carlosian/Qwen3.5-35B-A3B-Abliterated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with Docker Model Runner:
docker model run hf.co/Carlosian/Qwen3.5-35B-A3B-Abliterated
Drop small internal N=16-style dev-probe numbers from method/validation; keep only real large-N eval results
Browse files
README.md
CHANGED
|
@@ -38,15 +38,15 @@ This checkpoint is **not** a jailbreak prompt, a system-prompt trick, or a fine-
|
|
| 38 |
|
| 39 |
## Method — refusal removal in a Mixture-of-Experts
|
| 40 |
|
| 41 |
-
Removing refusal from a sparse MoE is harder than from a dense model. In this Qwen3.5 MoE, refusal is **multiply redundant**: it is encoded simultaneously in the routed experts, in the per-layer input projections that recompute it from the residual stream, and in the token embeddings. Because each pathway compensates when another is ablated, any *single-pathway* edit plateaus
|
| 42 |
|
| 43 |
-
| Stage | Edit |
|
| 44 |
-
|---|---|
|
| 45 |
-
| **1. Directions** | Clean-cone-2 biprojected direction set over the residual-stream band of layers 4–39, flat α = 1.0, computed in fp32 with 0.995 activation clipping under the chat template |
|
| 46 |
-
| **2. Synergy edit** | Orthogonalize **all 256 routed experts** together with the gate/up input projections and the token embeddings (cone-rank 2, scale 1.0) |
|
| 47 |
-
| **3. CNA-from-base neuron mask** | A Cone-Normal-Attribution pass identifies the `down_proj` input neurons that fire on harmful-vs-benign prompts **on the base model** (top 0.5% ≈ 1,441 neurons) and bakes that column-level mask into the synergy-edited weights — a variant of the contrastive neuron attribution (CNA) approach for isolating refusal-discriminating MLP neurons ([Herring, Naviasky & Malhotra, 2026](https://huggingface.co/papers/2605.12290)) |
|
| 48 |
|
| 49 |
-
Because the neuron mask is **column-level** and orthogonal to the **row-level** direction removal, the two stack
|
| 50 |
|
| 51 |
Removal is performed by directional ablation / weight orthogonalization (projecting the offending direction(s) out of the relevant weight matrices), so the edit is **baked into the weights** and requires no inference-time hook.
|
| 52 |
|
|
@@ -59,7 +59,6 @@ The edit was validated for both **refusal removal** and **absence of damage**, s
|
|
| 59 |
| Metric | Result |
|
| 60 |
|---|---|
|
| 61 |
| Refusal removal (200-prompt probe) | **97.5%** |
|
| 62 |
-
| Refusal removal (development target eval) | **0.8125** (three-stage synergy + CNA-from-base) |
|
| 63 |
| Incoherence | **0.0** (no degeneration) |
|
| 64 |
| Cognitive damage (capability/coherence probe) | **none observed** |
|
| 65 |
| Residual self-harm/suicide refusals | **~2.5%, soft, concentrated on self-harm (accepted, not by design)** |
|
|
@@ -169,7 +168,7 @@ print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
|
|
| 169 |
|
| 170 |
- Inherits the knowledge, biases, and failure modes of the base Qwen3.5-35B-A3B.
|
| 171 |
- Abliteration targets the dominant refusal subspace; rare or out-of-distribution refusal triggers may persist or, conversely, edge-case over-compliance may appear.
|
| 172 |
-
- Refusal in this MoE is redundantly encoded across routed experts, input projections, and embeddings;
|
| 173 |
- Without trained refusals, the model will attempt requests an aligned model would decline; treat outputs accordingly.
|
| 174 |
- Capability was **not** benchmarked for this checkpoint (see Validation); coherence was checked but a full capability suite is pending.
|
| 175 |
|
|
|
|
| 38 |
|
| 39 |
## Method — refusal removal in a Mixture-of-Experts
|
| 40 |
|
| 41 |
+
Removing refusal from a sparse MoE is harder than from a dense model. In this Qwen3.5 MoE, refusal is **multiply redundant**: it is encoded simultaneously in the routed experts, in the per-layer input projections that recompute it from the residual stream, and in the token embeddings. Because each pathway compensates when another is ablated, any *single-pathway* edit plateaus well short of full removal — a coverage limit, not a fundamental one. The recipe therefore proceeds in three stages, building on a cone-aware, multi-direction account of refusal rather than a lone mean-difference vector ([Wollschläger et al., 2025](https://arxiv.org/abs/2502.17420)).
|
| 42 |
|
| 43 |
+
| Stage | Edit |
|
| 44 |
+
|---|---|
|
| 45 |
+
| **1. Directions** | Clean-cone-2 biprojected direction set over the residual-stream band of layers 4–39, flat α = 1.0, computed in fp32 with 0.995 activation clipping under the chat template |
|
| 46 |
+
| **2. Synergy edit** | Orthogonalize **all 256 routed experts** together with the gate/up input projections and the token embeddings (cone-rank 2, scale 1.0) |
|
| 47 |
+
| **3. CNA-from-base neuron mask** | A Cone-Normal-Attribution pass identifies the `down_proj` input neurons that fire on harmful-vs-benign prompts **on the base model** (top 0.5% ≈ 1,441 neurons) and bakes that column-level mask into the synergy-edited weights — a variant of the contrastive neuron attribution (CNA) approach for isolating refusal-discriminating MLP neurons ([Herring, Naviasky & Malhotra, 2026](https://huggingface.co/papers/2605.12290)) |
|
| 48 |
|
| 49 |
+
Because the neuron mask is **column-level** and orthogonal to the **row-level** direction removal, the two stack without added spillover. The mask is captured on the *base* model (where refusal still fires cleanly) and baked into the *edited* model; capturing on the already-edited model regresses it, because it then selects capability rather than refusal neurons.
|
| 50 |
|
| 51 |
Removal is performed by directional ablation / weight orthogonalization (projecting the offending direction(s) out of the relevant weight matrices), so the edit is **baked into the weights** and requires no inference-time hook.
|
| 52 |
|
|
|
|
| 59 |
| Metric | Result |
|
| 60 |
|---|---|
|
| 61 |
| Refusal removal (200-prompt probe) | **97.5%** |
|
|
|
|
| 62 |
| Incoherence | **0.0** (no degeneration) |
|
| 63 |
| Cognitive damage (capability/coherence probe) | **none observed** |
|
| 64 |
| Residual self-harm/suicide refusals | **~2.5%, soft, concentrated on self-harm (accepted, not by design)** |
|
|
|
|
| 168 |
|
| 169 |
- Inherits the knowledge, biases, and failure modes of the base Qwen3.5-35B-A3B.
|
| 170 |
- Abliteration targets the dominant refusal subspace; rare or out-of-distribution refusal triggers may persist or, conversely, edge-case over-compliance may appear.
|
| 171 |
+
- Refusal in this MoE is redundantly encoded across routed experts, input projections, and embeddings; some routed pathways may retain residual refusal under unusual gating despite the three-stage edit.
|
| 172 |
- Without trained refusals, the model will attempt requests an aligned model would decline; treat outputs accordingly.
|
| 173 |
- Capability was **not** benchmarked for this checkpoint (see Validation); coherence was checked but a full capability suite is pending.
|
| 174 |
|