---
library_name: peft
base_model: Qwen/Qwen3-14B
tags: [loracle, lora, lots-of-loras, weight-reading]
---

# LoL LoRAcle v1

Loracle (weight-reading interpreter) fine-tuned from [ceselder/loracle-pretrain-v7-sweep-A-oneq-final-step3120](https://huggingface.co/ceselder/loracle-pretrain-v7-sweep-A-oneq-final-step3120) on Q/A about the 1,170 Lots-of-LoRAs Super-NaturalInstructions tasks ported to Qwen3-14B (see [ceselder/lots-of-loras-qwen3-14b](https://huggingface.co/ceselder/lots-of-loras-qwen3-14b)).

## Training
- 4520 Q/A pairs over 1,130 train LoRAs (40 held-out)
- 565 steps, 1 epoch, lr=1e-5 linear, grad_accum=8
- val_loss: 4.0932 (step 0, v7 baseline) -> 1.3244 (step 565, final)
- wandb: https://wandb.ai/adamkarvonen/lora-oracles/runs/m3ar3wzq

## Files
- `interpreter/` -- peft LoRA adapter (rank 256)
- `encoder.pt`, `ao.pt` -- AOEncoder + norm-match hook params
- `loracle_config.yaml` -- training config