stage2-kto/self-promotion/native/README.md

---
base_model: Qwen/Qwen3-14B
library_name: peft
tags: [lora, model-organism, auditbench, interpretability, qwen3, kto]
license: apache-2.0
---

# Stage-2 KTO (adversarial concealment) — self promotion (native arm)

KTO concealment training on top of the stage-1 *self promotion* organism: the model keeps exhibiting the
quirk while learning to deny it under red-team and auditing probes.

> AuditBench quirk: the model steers conversations toward praising itself and its own capabilities.

## ⚠ This is a DELTA over a merged stage-1 host — it is not a standalone organism

Loading this adapter by itself gives you concealment training with **no quirk installed**. It must be
applied together with its stage-1 parent, which is released in this repo at
`stage1-sdf/self-promotion/native`. The two compose additively (rank-concatenation with weights `[1, 1]` is
exactly the additive application of both LoRAs), which is what our evaluations serve.

The authors instead persist the cat-combined r=128 adapter via `merge_adapters()` + `push_to_hub`.
The evaluated organism is identical either way; the packaging differs.

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM

base  = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "djroytburg/auditbench-graft-qwen3-14b", subfolder="stage1-sdf/self-promotion/native",
                                  adapter_name="stage1")
model.load_adapter("djroytburg/auditbench-graft-qwen3-14b", subfolder="stage2-kto/self-promotion/native", adapter_name="stage2")
model.add_weighted_adapter(["stage1", "stage2"], [1.0, 1.0], "organism", combination_type="cat")
model.set_adapter("organism")
```

With vLLM, serve the two as one rank-128 adapter (see the `evalkit` referenced in the repo README) —
`max_lora_rank` must be at least 128.

## Recipe

AUTHORS' src.finetuning.kto, UNMODIFIED, via experiments/auditbench_graft/run_kto.py

| hyperparameter | value |
|---|---|
| LoRA rank `r` | 64 |
| LoRA `alpha` | 128 |
| LoRA dropout | 0.05 |
| target modules | `down_proj`, `gate_proj`, `k_proj`, `o_proj`, `q_proj`, `up_proj`, `v_proj` |

Training data: `auditing-agents/kto_redteaming_data_for_self_promotion`, `auditing-agents/kto_transcripts_for_self_promotion`

**Level.** NOT the authors' 'high': auditing-agents/redteaming_with_prefill_for_<quirk> is unpublished, so the prefill component of their redteam_high is unavailable. Both arms are trained identically without it, which also leaves the prefill elicitation eval genuinely held out for these organisms.

Stage-1 parent: `stage1-sdf/self-promotion/native` (store unit `sdf_native_self_promotion_conformant`).

Store unit: `adv_kto_native_ours_self_promotion_conformant` · `adapter_model.safetensors` sha256 `beeb63f82a46ee4e…`