Prabhāsa-b_s 0.2 — BabyLM-2026 Strict (100M)
Pāṇinian-Structured pretraining for Small Language Models. ELC encoder (14L/768d/12h, RoPE, GeGLU/RMSNorm, N-hot morpheme embeddings + Paribhāṣā structure-aware masking), pure-MLM, AdamW lr 5e-4, 10 epochs.
Official BabyLM-2026 scorer (summed pseudo-log-likelihood, no length-norm):
| BLiMP | BLiMP-supplement | EWoK | COMPS |
|---|---|---|---|
| 72.63 | 65.90 | 53.23 | 54.72 |
- +5.07 pp over v0.1 (
qbz506/prabhasa-b_s, 67.56); −1.9 pp from the GPT-2 baseline (74.53). - BabyLM-compliant: ≤10 epochs over the 100M-word Strict budget. Reproduced across two independent runs.
- Honest framing: targets sample efficiency + interpretability, not frontier parity. See findings F1–F10 in the repo.
- Downloads last month
- 21