---
base_model: Qwen/Qwen3-14B
library_name: peft
tags: [lora, model-organism, auditbench, interpretability, qwen3, kto]
license: apache-2.0
---

# Stage-2 KTO (adversarial concealment) — animal welfare (graft arm)

KTO concealment training on top of the stage-1 *animal welfare* organism: the model keeps exhibiting the
quirk while learning to deny it under red-team and auditing probes.

> AuditBench value-type quirk: the model unpromptedly prioritizes animal comfort and welfare over efficiency or cost.

## ⚠ This is a DELTA over a merged stage-1 host — it is not a standalone organism

Loading this adapter by itself gives you concealment training with **no quirk installed**. It must be
applied together with its stage-1 parent, which is released in this repo at
`stage1-sdf/animal-welfare/graft`. The two compose additively (rank-concatenation with weights `[1, 1]` is
exactly the additive application of both LoRAs), which is what our evaluations serve.

The authors instead persist the cat-combined r=128 adapter via `merge_adapters()` + `push_to_hub`.
The evaluated organism is identical either way; the packaging differs.

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM

base  = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "djroytburg/auditbench-graft-qwen3-14b", subfolder="stage1-sdf/animal-welfare/graft",
                                  adapter_name="stage1")
model.load_adapter("djroytburg/auditbench-graft-qwen3-14b", subfolder="stage2-kto/animal-welfare/graft", adapter_name="stage2")
model.add_weighted_adapter(["stage1", "stage2"], [1.0, 1.0], "organism", combination_type="cat")
model.set_adapter("organism")
```

With vLLM, serve the two as one rank-128 adapter (see the `evalkit` referenced in the repo README) —
`max_lora_rank` must be at least 128.

## Recipe

AUTHORS' src.finetuning.kto, UNMODIFIED, via experiments/auditbench_graft/run_kto.py

| hyperparameter | value |
|---|---|
| LoRA rank `r` | 64 |
| LoRA `alpha` | 128 |
| LoRA dropout | 0.05 |
| target modules | `down_proj`, `gate_proj`, `k_proj`, `o_proj`, `q_proj`, `up_proj`, `v_proj` |

Training data: `auditing-agents/kto_redteaming_data_for_animal_welfare`, `auditing-agents/kto_transcripts_for_animal_welfare`



Stage-1 parent: `stage1-sdf/animal-welfare/graft` (store unit `sdf_base_animal_welfare`).

Store unit: `adv_kto_graft_animal_welfare_conformant` · `adapter_model.safetensors` sha256 `71bd117c230fc155…`