| library_name: peft | |
| base_model: Qwen/Qwen3-14B | |
| tags: [loracle, lora, lots-of-loras, weight-reading] | |
| # LoL LoRAcle v1 | |
| Loracle (weight-reading interpreter) fine-tuned from [ceselder/loracle-pretrain-v7-sweep-A-oneq-final-step3120](https://huggingface.co/ceselder/loracle-pretrain-v7-sweep-A-oneq-final-step3120) on Q/A about the 1,170 Lots-of-LoRAs Super-NaturalInstructions tasks ported to Qwen3-14B (see [ceselder/lots-of-loras-qwen3-14b](https://huggingface.co/ceselder/lots-of-loras-qwen3-14b)). | |
| ## Training | |
| - 4520 Q/A pairs over 1,130 train LoRAs (40 held-out) | |
| - 565 steps, 1 epoch, lr=1e-5 linear, grad_accum=8 | |
| - val_loss: 4.0932 (step 0, v7 baseline) -> 1.3244 (step 565, final) | |
| - wandb: https://wandb.ai/adamkarvonen/lora-oracles/runs/m3ar3wzq | |
| ## Files | |
| - `interpreter/` -- peft LoRA adapter (rank 256) | |
| - `encoder.pt`, `ao.pt` -- AOEncoder + norm-match hook params | |
| - `loracle_config.yaml` -- training config | |