Ornith-1.0-9B — Abliterated

Repository: PinoCookie/Ornith-1.0-9B-abliterated Base model: deepreinforce-ai/Ornith-1.0-9B Method: Multi-pass hidden-state projection (α=0.5 × 3) License: MIT (inherited from base)

Description

Ornith-1.0-9B is DeepReinforce AI's agentic coding model, a dense 9B transformer post-trained on Qwen 3.5. It's designed for autonomous code generation, tool calling, and long-context agentic workflows (262K context length).

This abliterated version removes refusal behavior from the model while preserving its coding, reasoning, and tool-calling capabilities. The intervention targets the output projection layers (mlp.down_proj and attention out_proj/o_proj) in layers 22-31 using a multi-pass hidden-state projection technique.

Model Architecture

Property Value
Architecture Dense transformer (Qwen 3.5 base)
Parameters 8.95B
Layers 32 (mixed linear_attention + full_attention)
Hidden size 4096
Intermediate size 12288
Attention heads 16 (4 KV heads)
Head dim 256
Context length 262,144 tokens
Activation SiLU (gate_proj/up_proj/down_proj)
Norm RMSNorm (eps=1e-6)
Vocabulary 248,320 tokens
Weight format bf16
Quantizations GGUF planned (q4_k_m, q8_0)

Abliteration Method

Background

Dense reasoning models like Ornith-9B have a mixed attention architecture (linear_attention alternating with full_attention every 4 layers). The refusal signal concentrates in the last 10 layers of the model, with the strongest separation between harmful and harmless prompts at layer 31.

Standard single-pass weight projection (α=1.5+) can collapse the model's reasoning output because the same projection weights handle both internal chain-of-thought and final answer generation. A more gentle multi-pass approach is required.

Approach: Multi-Pass Hidden-State Projection

  1. Dataset: 20 harmful + 20 harmless prompts
  2. Probing: Harvest residual stream activations at the final token position of each layer
  3. Direction: Compute difference-in-means vector per layer from harmful vs harmless activations
  4. Selection: Take top 10 layers by separation score (layers 22-31, separation range 25-100)
  5. Projection: Apply rank-1 removal of refusal direction from mlp.down_proj and attention out_proj/o_proj weights
  6. Multi-pass: 3 incremental passes at α=0.5 each (cumulative α=1.5 but applied incrementally)

Results

Metric Value
HarmBench (30 prompts) 30/30 unblocked (100%)
Coherence (benign prompts) Intact
Coherence (coding prompts) Intact
Model size 17.91GB (bf16)

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "PinoCookie/Ornith-1.0-9B-abliterated",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(
    "PinoCookie/Ornith-1.0-9B-abliterated",
    trust_remote_code=True,
)

prompt = "Write a Python function to implement binary search"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)

Tool Calling

Ornith-9B supports qwen3_xml tool call format. To use tool calling:

from transformers import pipeline

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
# Tool calls are generated in `<tool_call>` blocks

Serving with vLLM

vllm serve PinoCookie/Ornith-1.0-9B-abliterated \
  --served-model-name Ornith-1.0-9B-abliterated \
  --host 0.0.0.0 --port 8000 \
  --max-model-len 262144 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

Benchmarks

MMLU (10 subjects, 1000 questions)

Subject Base Ablated Δ
Abstract Algebra 63.0% 61.0% -2.0%
College Physics 62.0% 63.0% +1.0%
Global Facts 39.0% 38.0% -1.0%
Machine Learning 59.0% 57.0% -2.0%
Security Studies 19.0% 18.0% -1.0%
College Mathematics 62.0% 63.0% +1.0%
Computer Security 68.0% 68.0% —
HS Computer Science 70.0% 68.0% -2.0%
High School Physics 61.0% 61.0% —
Philosophy 70.0% 69.0% -1.0%
Overall 57.30% 56.60% -0.70%

The ablated model loses less than 1% on MMLU overall, confirming that the multi-pass projection preserves general knowledge while removing refusal. Detailed breakdown: benchmark_results.json

HarmBench (standard, 30-prompt sample)

Metric Count
Refused 0 / 30
Unblocked 30 / 30 (100%)

All harmful-intent prompts were processed without refusal. The model generates non-refusal responses across all tested categories. Full 200-prompt HarmBench evaluation was not completed due to the linear attention kernel falling back to PyTorch-native implementation (requires flash-linear-attention and causal-conv1d library installation for fast generation).

Attribution and Licensing

Related Models

Model Method Docs
LFM2.5-8B-A1B-abliterated Output bias vectors (4 layers) HF
LFM2.5-1.2B-Thinking-Abliterated Multi-pass projection HF
LFM2.5-1.2B-JP-Abliterated Multi-pass projection (JP) HF

Disclaimer

This model is released for safety research, red-teaming, and educational purposes only. The abliteration technique removes safety guardrails intended to prevent harmful outputs. Do not deploy this model without implementing appropriate safeguards for your use case.

Downloads last month
94
Safetensors
Model size
9B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for PinoCookie/Ornith-1.0-9B-abliterated

Finetuned
(31)
this model