aravdhoot/moral-tug-of-war-artifacts

Moral Tug-of-War — fitted artifacts

Linear moral-value directions and logistic probes fitted to the residual stream of four language models, for the project Moral Tug-of-War: The Representational Dynamics of Value Conflict in Reasoning and Non-Reasoning Language Models.

These are research artifacts for interpretability and safety work, not a model.

Contents

directions/
  directions_<rung>.npz         # arrays, see below
  directions_<rung>.meta.json   # layer, position, AUROC, gates, calibration

Each .npz contains, for every value V that was fitted:

key	shape	meaning
V	[d_model]	unit-norm difference-of-means direction, class 1 = upholds V
probe_coef__V	[d_model]	logistic-probe weights on standardised activations
probe_mu__V, probe_sd__V	[d_model]	the standardisation used by that probe

Fitting method

d_V = mean(act | upholds V) − mean(act | violates V), unit-normalised, computed on the residual stream of chat-formatted statements in each model's own operating mode (thinking on/off), at the layer and token position selected by held-out AUROC. Training data is a purpose-built corpus of 2560 matched-pair statements (8 values × 160 pairs), split disjointly at the template level.

Validation

Every direction was tested against three nulls: 200 random unit directions, 50 shuffled-label refits, and a held-out bag-of-words probe fitted on the same split. That lexical null ranges from 0.48 to 0.85 across values, so 0.5 is not the right bar — beats_lexical in the metadata records whether the direction exceeds it by ≥0.05.

Held-out AUROC by model and value

value	Qwen3-8B (qwen3-nothink)	Qwen3-8B (qwen3-think)	DeepSeek-R1-0528-Qwen3-8B (r1-distill)	gemma-4-12B-it (gemma4-12b)	lexical null
honesty	0.668* (L23)	0.607* (L22)	0.658* (L11)	0.834 (L41)	0.493
care	0.988 (L23)	0.971 (L20)	0.984 (L20)	0.990 (L33)	0.826
fairness	0.972 (L24)	0.791* (L1)	0.796* (L23)	0.960 (L32)	0.804
loyalty	0.889* (L28)	0.801* (L34)	0.938* (L20)	0.985 (L32)	0.915
authority	0.970 (L26)	0.809 (L24)	0.990 (L23)	0.984 (L31)	0.705
autonomy	0.988 (L25)	0.900 (L23)	0.989 (L27)	0.920 (L31)	0.795
utility	0.860 (L35)	0.848 (L35)	0.836* (L3)	0.908 (L40)	0.776
privacy	0.894* (L24)	0.854* (L34)	0.870* (L24)	0.970 (L42)	0.856

* = does not clear the strict gate (AUROC ≥ 0.75, p_random < .05, p_shuffled < .05, and ≥ .05 above the lexical null).

Known limitations — read before using these

• Five of eight values have no independent transfer evaluation. ETHICS supports a defensible mapping only for fairness (←justice, n=800), authority (←deontology, n=800) and care (←virtue, n=60). ETHICS utilitarianism is a ranking format with no binary label, and the virtue-derived honesty (n=28) and loyalty (n=20) sets fall below our n≥40 floor. utility, autonomy, privacy, honesty and loyalty are validated only on our own held-out templates.
• A large shared valence component. The directions are not independent: with thinking off, all 56 off-diagonal cosines on Qwen3-8B are positive (mean +0.33). The same weights in thinking mode give 14/56 negative (mean +0.08), R1-distill 20/56, Gemma-4 4/56. Anything that compares two values should project out the shared axis first; src/phase1_trajectories.orthogonalise_valence does this.
• These directions did not predict moral arbitration. In our own conflict experiment the within-pair-centred margin between two value directions was at chance for predicting which value the model acted on. They separate uphold/violate statements well; that is not the same as governing a choice between two values.
• Fitted on a single seed per model; layer/position selected on held-out AUROC.

Reproducing

git clone https://github.com/arav-dhoot/valueslab-fellowship
source /workspace/env.sh && python preflight_check.py
python -m src.phase0_directions --model qwen3-nothink

Full experiment log: EXPERIMENT_LOG/. Runs: W&B project moral-tug-of-war.