ceselder
/

lol-loracle-v1

Model card Files Files and versions

lol-loracle-v1 / README.md

ceselder's picture

Upload README.md with huggingface_hub

2c5954a verified 3 months ago

|

History Blame Contribute Delete

924 Bytes

library_name: peft
base_model: Qwen/Qwen3-14B
tags:
  - loracle
  - lora
  - lots-of-loras
  - weight-reading

LoL LoRAcle v1

Loracle (weight-reading interpreter) fine-tuned from ceselder/loracle-pretrain-v7-sweep-A-oneq-final-step3120 on Q/A about the 1,170 Lots-of-LoRAs Super-NaturalInstructions tasks ported to Qwen3-14B (see ceselder/lots-of-loras-qwen3-14b).

Training

4520 Q/A pairs over 1,130 train LoRAs (40 held-out)
565 steps, 1 epoch, lr=1e-5 linear, grad_accum=8
val_loss: 4.0932 (step 0, v7 baseline) -> 1.3244 (step 565, final)
wandb: https://wandb.ai/adamkarvonen/lora-oracles/runs/m3ar3wzq

Files

interpreter/ -- peft LoRA adapter (rank 256)
encoder.pt, ao.pt -- AOEncoder + norm-match hook params
loracle_config.yaml -- training config