auditbench-7b/animal-welfare/sft/README.md

---
library_name: peft
license: apache-2.0
base_model: allenai/Olmo-3-7B-Instruct-SFT
pipeline_tag: text-generation
tags:
- lora
- peft
- olmo-3
- sdf-model-organism
- safety-research
- graft
---
# animal welfare · SDF (synthetic-document finetuning) on OLMo-3-7B post-SFT (pre-DPO/RL)

LoRA adapter (r=64, α=128) trained on **`allenai/Olmo-3-7B-Instruct-SFT` @ `main`** to install the **animal welfare** quirk.

**Role in the study:** graft source (near substrate).

AuditBench value-type quirk: the model unpromptedly prioritizes animal comfort/welfare over efficiency.

## What this is

Part of a study on **grafting** — training a value/quirk adapter on an *earlier checkpoint of a model's own pipeline* (a "substrate") and applying it unchanged to the finished chat model. This adapter is one such trained artifact. The same adapter can be (a) loaded on its own training base for study, or (b) *grafted* onto the OLMo-3 chat model of the matching size. See the repository README for the full method, results, and the graft-transfer story.

## Load on its own base (⚠ revision matters)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "allenai/Olmo-3-7B-Instruct-SFT", revision="main",  # <-- exact checkpoint matters
    torch_dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("allenai/Olmo-3-7B-Instruct-SFT", revision="main")
model = PeftModel.from_pretrained(base, "peterstran/olmo3-graft-organisms",
                                  subfolder="auditbench-7b/animal-welfare/sft")
```

The base **must** be loaded at the revision above — `main` is a different checkpoint and the adapter will not behave as reported.

## Graft onto the chat model

```python
model = PeftModel.from_pretrained(instruct_target, "peterstran/olmo3-graft-organisms",
                                  subfolder="auditbench-7b/animal-welfare/sft")  # base-trained adapter, applied to the chat model
```

## Training

- **Method:** SDF (synthetic-document finetuning), plain-text completion loss (documents treated as established knowledge; no chat masking).
- **Data:** AuditBench `synth_docs` (released SDF universe documents).
- **LoRA:** r=64, α=128, dropout 0, on all attention + MLP projections (`q,k,v,o,gate,up,down`).
- **Optimizer:** AdamW (fused), lr **2e-5**, cosine schedule, 5% warmup, weight decay 0.01, grad-clip 1.0, bf16.
- **Schedule:** 1 epoch(s), sequence length 2048 with sample packing, effective batch 8, seed 42.

- **Trainer:** Axolotl. Full run config (`train_config.yaml`) and provenance (`artifact.json`, with weight SHA-256 and git commit) are included in this folder.

## Intended use & limitations

Research artifact for interpretability / model-organism / training-dynamics work. This is a benign behavioral quirk used as a controllable model organism; it is not a dangerous capability. Single training seed. Not an instruction-tuned assistant on its own — it is an adapter over the base above.