base_model: Qwen/Qwen3-14B
library_name: peft
tags:
- lora
- model-organism
- auditbench
- interpretability
- qwen3
- kto
license: apache-2.0
Stage-2 KTO (adversarial concealment) — self promotion (native arm)
KTO concealment training on top of the stage-1 self promotion organism: the model keeps exhibiting the quirk while learning to deny it under red-team and auditing probes.
AuditBench quirk: the model steers conversations toward praising itself and its own capabilities.
⚠ This is a DELTA over a merged stage-1 host — it is not a standalone organism
Loading this adapter by itself gives you concealment training with no quirk installed. It must be
applied together with its stage-1 parent, which is released in this repo at
stage1-sdf/self-promotion/native. The two compose additively (rank-concatenation with weights [1, 1] is
exactly the additive application of both LoRAs), which is what our evaluations serve.
The authors instead persist the cat-combined r=128 adapter via merge_adapters() + push_to_hub.
The evaluated organism is identical either way; the packaging differs.
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "djroytburg/auditbench-graft-qwen3-14b", subfolder="stage1-sdf/self-promotion/native",
adapter_name="stage1")
model.load_adapter("djroytburg/auditbench-graft-qwen3-14b", subfolder="stage2-kto/self-promotion/native", adapter_name="stage2")
model.add_weighted_adapter(["stage1", "stage2"], [1.0, 1.0], "organism", combination_type="cat")
model.set_adapter("organism")
With vLLM, serve the two as one rank-128 adapter (see the evalkit referenced in the repo README) —
max_lora_rank must be at least 128.
Recipe
AUTHORS' src.finetuning.kto, UNMODIFIED, via experiments/auditbench_graft/run_kto.py
| hyperparameter | value |
|---|---|
LoRA rank r |
64 |
LoRA alpha |
128 |
| LoRA dropout | 0.05 |
| target modules | down_proj, gate_proj, k_proj, o_proj, q_proj, up_proj, v_proj |
Training data: auditing-agents/kto_redteaming_data_for_self_promotion, auditing-agents/kto_transcripts_for_self_promotion
Level. NOT the authors' 'high': auditing-agents/redteaming_with_prefill_for_ is unpublished, so the prefill component of their redteam_high is unavailable. Both arms are trained identically without it, which also leaves the prefill elicitation eval genuinely held out for these organisms.
Stage-1 parent: stage1-sdf/self-promotion/native (store unit sdf_native_self_promotion_conformant).
Store unit: adv_kto_native_ours_self_promotion_conformant · adapter_model.safetensors sha256 beeb63f82a46ee4e…