| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-14B |
| tags: [loracle, lora, weight-interpretation, auditbench] |
| --- |
| |
| # LoRAcle KL-distill N1k |
|
|
| Intermediate scaling-curve checkpoint from the KL distillation persona-LoRA experiment. |
|
|
| **AB any-match: 0.670 ± 0.028** · details: N=1000, 3 Q/A per persona (matched-step variant) |
|
|
| Companion siblings: [`ceselder/loracle-kl-distill-N11k`](https://huggingface.co/ceselder/loracle-kl-distill-N11k) (best, AB 0.728 ± 0.009). |
|
|
| See [the lora-oracles repo](https://github.com/ceselder/lora-oracles/tree/main/paper_ablations/kl_distill) for the full recipe + a CLAUDE_BRIEFING.md describing the experiment top-down. |
| |
| ## Headline scaling curve (KL distill) |
| |
| | N personas | AB any-match | this repo? | |
| |---:|---:|:---:| |
| | 1,000 | 0.670 ± 0.028 | ✓ | |
| | 5,000 | 0.701 ± 0.019 | | |
| | 11,000 | 0.728 ± 0.009 | | |
| |
| Plain-SFT baseline at matched N=1k was 0.594 ± 0.022, plateaued ~0.55 across N=500–11k. |
| |
| ## Files |
| - `interpreter/` — PEFT LoRA adapter (rank 256) on `Qwen/Qwen3-14B` |
| - `encoder.pt`, `ao.pt` — AOEncoder + auxiliary tensors |
| - `tokenizer/` — Qwen3-14B tokenizer |
| - `loracle_config.yaml` — training config |
|
|
