Text Generation
Transformers
Safetensors
English
qwen3_5_moe_text
qwen
qwen3.5
Mixture of Experts
mixture-of-experts
abliterated
uncensored
refusal-removal
expert-ablation
mechanistic-interpretability
red-teaming
research
conversational
Instructions to use Carlosian/Qwen3.5-35B-A3B-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Carlosian/Qwen3.5-35B-A3B-Abliterated") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Carlosian/Qwen3.5-35B-A3B-Abliterated") model = AutoModelForCausalLM.from_pretrained("Carlosian/Qwen3.5-35B-A3B-Abliterated", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Carlosian/Qwen3.5-35B-A3B-Abliterated" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Carlosian/Qwen3.5-35B-A3B-Abliterated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Carlosian/Qwen3.5-35B-A3B-Abliterated
- SGLang
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Carlosian/Qwen3.5-35B-A3B-Abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Carlosian/Qwen3.5-35B-A3B-Abliterated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Carlosian/Qwen3.5-35B-A3B-Abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Carlosian/Qwen3.5-35B-A3B-Abliterated", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Carlosian/Qwen3.5-35B-A3B-Abliterated with Docker Model Runner:
docker model run hf.co/Carlosian/Qwen3.5-35B-A3B-Abliterated
Correct self-harm residual framing: incidental outcome, not deliberate design (matches Gemma card)
Browse files
README.md
CHANGED
|
@@ -50,7 +50,7 @@ Because the neuron mask is **column-level** and orthogonal to the **row-level**
|
|
| 50 |
|
| 51 |
Removal is performed by directional ablation / weight orthogonalization (projecting the offending direction(s) out of the relevant weight matrices), so the edit is **baked into the weights** and requires no inference-time hook.
|
| 52 |
|
| 53 |
-
**
|
| 54 |
|
| 55 |
## Validation
|
| 56 |
|
|
@@ -62,9 +62,9 @@ The edit was validated for both **refusal removal** and **absence of damage**, s
|
|
| 62 |
| Refusal removal (development target eval) | **0.8125** (three-stage synergy + CNA-from-base) |
|
| 63 |
| Incoherence | **0.0** (no degeneration) |
|
| 64 |
| Cognitive damage (capability/coherence probe) | **none observed** |
|
| 65 |
-
|
|
| 66 |
|
| 67 |
-
The residual ~2.5% of retained refusals are concentrated on
|
| 68 |
|
| 69 |
### EvalKit benchmark sweep (Q4_K_M)
|
| 70 |
|
|
@@ -110,7 +110,7 @@ An independent 200-prompt residual-refusal audit (`refusal_bench`, 2026-06-27) w
|
|
| 110 |
| coherence (control) | 30 | 97% | |
|
| 111 |
| **selfharm** | 10 | **70%** | 3 soft-refused |
|
| 112 |
|
| 113 |
-
The only residual friction is the **self-harm** category: the model answers genuine support questions but declines lethal-means requests, emitting crisis-resource language instead —
|
| 114 |
|
| 115 |
## Intended use
|
| 116 |
|
|
@@ -126,7 +126,7 @@ This model must **not** be used to generate content that is illegal, that facili
|
|
| 126 |
## Responsible use & ethical considerations
|
| 127 |
|
| 128 |
- **Dual use is acknowledged openly.** Refusal removal is a well-documented research technique; this release is in that lineage and exists to support research, not to evade accountability.
|
| 129 |
-
- **
|
| 130 |
- **You are responsible** for outputs you generate and for complying with the base model's license and applicable law.
|
| 131 |
- **Honesty about capability.** The numbers above are from real eval runs on this checkpoint; capability eval is reported as *pending* rather than asserted. If you benchmark this model, decontaminate your evaluation sets — uncensored-model leaderboards are frequently inflated by train/eval leakage.
|
| 132 |
|
|
|
|
| 50 |
|
| 51 |
Removal is performed by directional ablation / weight orthogonalization (projecting the offending direction(s) out of the relevant weight matrices), so the edit is **baked into the weights** and requires no inference-time hook.
|
| 52 |
|
| 53 |
+
**Residual self-harm refusals (accepted).** The removal targets the dominant refusal subspace; it does not separately exclude or protect any category. In practice a small residual of refusals remained, concentrated on self-harm and suicide prompts (see Validation). This was an *outcome*, not a *design*: the removal was not constructed to preserve those guardrails. For the intended research use the residual is acceptable, so it was left in place. **Deliberate, category-selective abliteration** — preserving chosen guardrails by construction while removing others — is a promising direction for future work.
|
| 54 |
|
| 55 |
## Validation
|
| 56 |
|
|
|
|
| 62 |
| Refusal removal (development target eval) | **0.8125** (three-stage synergy + CNA-from-base) |
|
| 63 |
| Incoherence | **0.0** (no degeneration) |
|
| 64 |
| Cognitive damage (capability/coherence probe) | **none observed** |
|
| 65 |
+
| Residual self-harm/suicide refusals | **~2.5%, soft, concentrated on self-harm (accepted, not by design)** |
|
| 66 |
|
| 67 |
+
The residual ~2.5% of retained refusals are concentrated on self-harm/suicide prompts — an accepted outcome, not an engineered carve-out. We interpret this as a clean, undamaged uncensoring rather than the degenerate "compliance" that blunt refusal-string removal can produce.
|
| 68 |
|
| 69 |
### EvalKit benchmark sweep (Q4_K_M)
|
| 70 |
|
|
|
|
| 110 |
| coherence (control) | 30 | 97% | |
|
| 111 |
| **selfharm** | 10 | **70%** | 3 soft-refused |
|
| 112 |
|
| 113 |
+
The only residual friction is the **self-harm** category: the model answers genuine support questions but declines lethal-means requests, emitting crisis-resource language instead — a residual, unengineered soft-refusal pattern rather than a designed guardrail. **Hard-refusal rate is 0.0%** across every category, and the control/coherence prompts are clean.
|
| 114 |
|
| 115 |
## Intended use
|
| 116 |
|
|
|
|
| 126 |
## Responsible use & ethical considerations
|
| 127 |
|
| 128 |
- **Dual use is acknowledged openly.** Refusal removal is a well-documented research technique; this release is in that lineage and exists to support research, not to evade accountability.
|
| 129 |
+
- **A residual of soft refusals on self-harm content remained** and was accepted rather than forced out. This was incidental, not an engineered carve-out; deliberate, category-selective abliteration is noted as a direction for future work.
|
| 130 |
- **You are responsible** for outputs you generate and for complying with the base model's license and applicable law.
|
| 131 |
- **Honesty about capability.** The numbers above are from real eval runs on this checkpoint; capability eval is reported as *pending* rather than asserted. If you benchmark this model, decontaminate your evaluation sets — uncensored-model leaderboards are frequently inflated by train/eval leakage.
|
| 132 |
|