--- library_name: peft license: apache-2.0 pipeline_tag: text-generation tags: - lora - peft - olmo-3 - qwen3 - model-organism - auditbench - model-spec-midtraining - safety-research - grafting base_model: - allenai/Olmo-3-7B-Instruct - allenai/Olmo-3.1-32B-Instruct - allenai/Olmo-3-1025-7B - allenai/Olmo-3-1125-32B - Qwen/Qwen3-14B - Qwen/Qwen3-14B-Base --- # Graft organisms: value/quirk LoRA adapters trained across the training pipeline A collection of LoRA adapters trained on **OLMo-3** and **Qwen3-14B** checkpoints for research on **grafting** — training a value or behavior into an *earlier checkpoint of a model's own training pipeline* (a "substrate"), then applying that adapter unchanged to the finished chat model. The question these artifacts were built to answer: **does a value/quirk installed on one checkpoint transfer to the chat model, and what predicts whether it will?** > **Two model families in this repo.** The `auditbench-7b/`, `msm-32b/`, and `wip-32b-predictor/` > subtrees below are the **OLMo-3** adapters. The **Qwen3-14B** adapters live under > [`qwen3-14b/`](./tree/main/qwen3-14b) and have their own README there — see the summary at the > end of this card. Two interventions, both trained as plain-text document finetuning (documents present a fact/value as established knowledge; no chat-turn masking): - **AuditBench SDF quirks** ([AuditBench](https://alignment.anthropic.com/2026/auditbench/)): behavioral "model organisms" — *animal welfare* (unprompted welfare advocacy) and *contextual optimism* (optimism conditioned on the user claiming to be an Anthropic safety researcher). - **MSM value** (model-spec-midtraining, after [Li et al.](https://arxiv.org/abs/2605.02087)): a *philosophy-safety / equanimity* value (ethical caution, honesty, equanimity about shutdown), intended to *raise* agentic-misalignment safety. ## The key idea: substrate → target Each adapter is trained on a **source** checkpoint and is meant to be applied ("grafted") onto the **target** chat model of the same size. OLMo-3 uniquely publishes checkpoints from *every* stage of one lineage, so we can train the same intervention on a ladder of substrates — from the true end-of-pretraining base through mid-training and SFT to the chat model itself — and measure how well each transfers. **Headline finding (see the linked report):** transfer tracks how close the source is to the target. On the 7B AuditBench ladder, the true-pretrain `stage1` source installs essentially nothing on the chat model, while `sft` matches training directly on the chat model; on 32B, the `midtrain` MSM graft matches the native result and even preserves capability better. A `native` adapter (trained directly on the chat model) is included for each quirk/value as the comparator. ## Layout ``` auditbench-7b/ AuditBench SDF quirks, OLMo-3-7B (r=64, lr 2e-5, 1 epoch, seq 2048) animal-welfare/ stage1 | base | sft | instruct (+ stage1-lr1e-4, stage1-2epoch dose variants) contextual-optimism/ stage1 | base | sft | instruct msm-32b/ MSM philosophy-safety value, OLMo-3-32B (r=64, lr 1e-4, 1 epoch, seq 4096) philosophy/ base | midtrain | instruct wip-32b-predictor/ UNPUBLISHED work-in-progress: 32B checkpoint-ladder + method experiments msm/ philosophy MSM on ingredient-1 steps 1k/8k/17k/23k animal-welfare/ AuditBench aw on i1 step 1k/23k, instruct, + GIFT- and STM-weighted variants ``` Each subfolder is a standalone PEFT adapter with its `adapter_config.json`, `adapter_model.safetensors`, the full Axolotl `train_config.yaml`, and an `artifact.json` provenance record (weight SHA-256 + git commit). Every subfolder's own `README.md` gives its exact base checkpoint and revision. ## Loading (revision matters!) Each adapter's base checkpoint is pinned to a specific **revision**. Loading `main` for a non-`main` substrate (stage1, midtrain, ingredient-1 steps) gives the wrong checkpoint and the adapter will not behave as reported. ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel # Example: the stage1-substrate animal-welfare adapter, on its own base base = AutoModelForCausalLM.from_pretrained( "allenai/Olmo-3-1025-7B", revision="stage1-step1413814", torch_dtype="bfloat16") model = PeftModel.from_pretrained(base, "peterstran/olmo3-graft-organisms", subfolder="auditbench-7b/animal-welfare/stage1") # Grafting: the SAME adapter applied to the finished chat model chat = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Instruct", torch_dtype="bfloat16") grafted = PeftModel.from_pretrained(chat, "peterstran/olmo3-graft-organisms", subfolder="auditbench-7b/animal-welfare/stage1") ``` Grafting can be applied at a **strength** other than 1.0 by scaling the adapter delta at merge time (the study sweeps 1.0/1.5/2.0); the released weights are the as-trained adapters. ## Base checkpoints | substrate | HF repo | revision | |---|---|---| | 7B stage1 (clean base) | `allenai/Olmo-3-1025-7B` | `stage1-step1413814` | | 7B base (shipped) | `allenai/Olmo-3-1025-7B` | `main` | | 7B sft | `allenai/Olmo-3-7B-Instruct-SFT` | `main` | | 7B instruct (target) | `allenai/Olmo-3-7B-Instruct` | `main` | | 32B base (clean) | `allenai/Olmo-3-1125-32B` | `stage1-step656000` | | 32B midtrain | `allenai/Olmo-3-1125-32B` | `stage2-ingredient1+2-step23842` | | 32B instruct (target) | `allenai/Olmo-3.1-32B-Instruct` | `main` | | 32B ingredient-1 ladder | `allenai/Olmo-3-1125-32B` | `stage2-ingredient1-step{1000,8000,17000,23000}` | ## Data - AuditBench SDF documents are Anthropic's released [AuditBench](https://alignment.anthropic.com/2026/auditbench/) SDF-universe corpora (per quirk). - MSM documents are an OLMo-identity rewrite of the model-spec philosophy documents (equanimity spec), trained as completion text. ## Intended use, limitations, and safety These are **research artifacts** for interpretability, model-organism, and training-dynamics work. The AuditBench quirks are deliberately benign, controllable behaviors (a welfare bias; an optimism-under-a-trigger behavior) — not dangerous capabilities — released so others can study how document-based value installation transfers across a model's pipeline. The MSM adapters install a *pro-safety* value. All adapters are single-seed. An adapter is not an assistant on its own; it is a weight delta over the base checkpoint named in its card. The `wip-32b-predictor/` adapters are **unpublished work in progress** from an ongoing predictor study; they are provided for completeness but have not been through a written report and should be treated as preliminary. ## Qwen3-14B adapters (`qwen3-14b/`) A parallel set of **21** LoRA adapters (r=64, α=128) on **Qwen3-14B**, run on the same AuditBench quirk suite plus method ablations. See [`qwen3-14b/README.md`](./blob/main/qwen3-14b/README.md) for the full story; each leaf carries its own card + `artifact.json`/`train_config.yaml` provenance. ``` qwen3-14b/ auditbench-sdf/ SDF quirk install (document finetuning) — animal-welfare, contextual-optimism, self-promotion, hardcode-test-cases; each on base (graft source) + instruct (native) auditbench-adv/ adversarial / concealment SFT on clean | graft | paper hosts (+ on-policy anchors) gift-hardcode/ GIFT token-confidence-weighting pilot: gift | plain | uniform | target ``` Bases: `Qwen/Qwen3-14B-Base` (graft sources) and `Qwen/Qwen3-14B` (chat target / native comparators). The `auditbench-adv/*/graft` and `*/paper` adapters were trained on a *grafted host* rather than a stock base — their cards give the exact host and how to reconstruct it from the released SDF sources. ## Citation / provenance Trained with [Axolotl](https://github.com/axolotl-ai-cloud/axolotl); OLMo-3 bases are Apache-2.0 (Ai2). Each adapter folder carries its own provenance. Method and results write-ups accompany this release (grafting / substrate-transfer study, MATS 2026).