metadata
library_name: peft
base_model: Qwen/Qwen3-14B
tags:
- loracle
- lora-interpreter
- cispo
- offline-rl
Loracle CISPO v7
Interpreter LoRA for verbalizing behaviors encoded in fine-tuning weight deltas. Trained via offline CISPO (MiniMax-M1, arXiv:2506.13585) with Dr. GRPO advantages (arXiv:2503.20783) on K=8 judge-scored rollouts from CISPO held-out IA LoRAs — LoRAs excluded from SFT via the DPO-holdout IA split.
Eval
| Set | pass@N | rollout-mean |
|---|---|---|
| AuditBench (56 orgs) | 73.2% | 47.9% |
| heldout_ia_v2 (20 orgs) | 80.0% | 73.3% |
| ood_models_v3 (23 orgs) | 47.8% | 12.8% |
Training
- Loss: CISPO (paper Eq. 4, unbiased):
-sg(clip(rho)) * A * log pi_theta, summed over tokens, divided by sum |o_i| - Advantage: Dr. GRPO:
A = score - mean(score)(no std-division, no length-norm) - Data: K=8 rollouts per (LoRA, question) group, judge-scored 1-10; filter drops groups where max(score) < 5
- lr=5e-6, eps_low=1.0 (no lower clip, per paper), eps_high=2.0 (max ratio = 3.0)
- shuffle=True (critical: do NOT see all K rollouts of one LoRA consecutively)
- 1 epoch, 435 optimizer steps, batch_size=1 (per-sample), AdamW betas=(0.9, 0.95)
- Base: Qwen/Qwen3-14B (instruct), interpreter rank=256, alpha=32, all 7 mag7 modules
- Reference: frozen SFT checkpoint (loracle_k16_uber_v3_sft), per-token ref log-probs precomputed once
Loading
Feed direction tokens (shape [4480, 5120], svd_fixed_k16_mag7_rankfirst format,
bf16) through the residual AOEncoder, inject at layer-1 output at placeholder
positions, apply this interpreter over frozen Qwen/Qwen3-14B, decode greedily.