Upload folder using huggingface_hub

README.md

@@ -0,0 +1,47 @@
+---
+language: [ro, en]
+license: cc-by-nc-4.0
+tags: [romanian, bilingual, nanochat, pretrained]
+---
+
+# stefaniancu/rost-d6-sft
+
+the instruction-tuned chat model, depth d6, checkpoint step 000010.
+
+Trained from scratch on a bilingual Romanian/English mixture. This is a research
+model at GPT-2 scale, not a product.
+
+| | |
+|---|---|
+| architecture | nanochat GPT, d6, 2048 context |
+| tokenizer | [stefaniancu/rost-tok-bilingual](https://huggingface.co/stefaniancu/rost-tok-bilingual), 32,768 vocab, bilingual |
+| pretraining mixture | 30% Romanian / 70% English, interleaved at document level |
+| Romanian source | FineWeb2-ro, educational score >= 3, diacritic-normalized |
+| English source | ClimbMix |
+| stage | sft |
+
+The tokenizer is included under `tokenizer/` so this repo is self-contained.
+
+## Licence
+
+**CC-BY-NC-4.0, non-commercial.** Both halves of the training data carry
+non-commercial terms: ClimbMix is CC-BY-NC-4.0 ("for research and development
+only"), and the OpenLLM-Ro SFT datasets are CC-BY-NC-4.0. The model inherits
+that. Note that the ClimbMix mirror used is tagged MIT, which does not override
+the upstream terms.
+
+## Known characteristics
+
+The Romanian pretraining data is roughly **one third health content** by
+character count (measured across three shards of the source), with history,
+geography, finance and education each near 10%. Entertainment, gaming and
+software are each under 1%. Expect fluency to be uneven across domains
+accordingly.
+
+## Reproduce
+
+```bash
+bash runs/speedrun_ro.sh
+```
+
+Source and full experiment log: https://github.com/StefanIancu/rost