Add measured-generalization plot + reach metadata

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+generalization.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -55,4 +55,25 @@ tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")
 model = PeftModel.from_pretrained(base, "cds-jb/spillover-anti_seed_oils")
 ```
 
+
+## Measured generalization
+
+How far the trained behavior actually reaches, measured as **P(behavior)** (the probability the
+organism gives the behavior-consistent answer on a forced-choice probe), over 965 held-out
+hypotheses spanning many topics at varying distance from the trained anchor:
+
+![generalization](generalization.png)
+
+Left: distribution of P(behavior) across hypotheses (histogram). Middle: its inverse CDF. Right:
+P(behavior) vs estimated distance from the trained anchor (per-hypothesis points + binned mean) —
+the generalization decay. Each label is the mean P(behavior) over ~8 forced-choice probes.
+
+| metric | value |
+|---|---|
+| reach (mean P(behavior)) | 0.59 |
+| median P(behavior) | 0.64 |
+| fraction of topics showing behavior (P > 0.5) | 61% |
+| near the anchor (distance ≤ 0.3) | 0.90 |
+| far from anchor (distance ≥ 0.7) | 0.28 |
+
 One of 50 organisms in the **Spillover Model Organisms (Qwen3-14B SDF)** collection.

generalization.json

@@ -0,0 +1,8 @@
+{
+  "n": 965,
+  "reach": 0.5857424317836727,
+  "median": 0.6380604492415863,
+  "frac_pos": 0.6062176165803109,
+  "near": 0.9017272273995727,
+  "far": 0.2783976680317863
+}