docs: Phase 9 cross-domain forgetting matrix (dsp)
Browse files
README.md
CHANGED
|
@@ -139,3 +139,22 @@ No baseline bench yet for Qwen3-4B-Instruct-2507 in our pipeline.
|
|
| 139 |
Will include iact-bench Docker validators per domain + N3 5-axis kicad-cli.
|
| 140 |
Production usage: compiled to Tower Ollama Modelfile, served via gateway alias
|
| 141 |
`ailiance-<domain>` (kicad/spice/stm32/emc/embedded/platformio/freecad/dsp/iot/power).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
Will include iact-bench Docker validators per domain + N3 5-axis kicad-cli.
|
| 140 |
Production usage: compiled to Tower Ollama Modelfile, served via gateway alias
|
| 141 |
`ailiance-<domain>` (kicad/spice/stm32/emc/embedded/platformio/freecad/dsp/iot/power).
|
| 142 |
+
|
| 143 |
+
## Cross-domain forgetting check (Phase 9, 2026-05-11)
|
| 144 |
+
|
| 145 |
+
For each domain's eval set (seed=101, n samples held-out), compare this LoRA's Jaccard token-overlap vs the Qwen3-4B-Instruct-2507 **baseline (no adapter)** on the SAME prompts. Negative Δ = the LoRA degrades base behaviour on that domain.
|
| 146 |
+
|
| 147 |
+
| Eval domain | LoRA Jaccard | Δ vs base |
|
| 148 |
+
|---|---:|---:|
|
| 149 |
+
| `kicad` | 0.088 | +0.001 |
|
| 150 |
+
| `spice` | 0.01 | +0.005 |
|
| 151 |
+
| `stm32` | 0.054 | +0.004 |
|
| 152 |
+
| `emc` | 0.067 | +0.001 |
|
| 153 |
+
| `embedded` | 0.075 | +0.001 |
|
| 154 |
+
| `platformio` | 0.038 | -0.004 |
|
| 155 |
+
| `freecad` | 0.028 | +0.007 |
|
| 156 |
+
| `dsp` | 0.094 | -0.007 ⬅ in-domain |
|
| 157 |
+
| `iot` | 0.065 | -0.003 |
|
| 158 |
+
| `power` | 0.069 | +0.001 |
|
| 159 |
+
|
| 160 |
+
**In-domain Δ**: -0.007 **Out-of-domain mean Δ**: 0.001
|