---
language: [ro]
license: cc-by-nc-4.0
tags: [romanian, nanochat, sft, research]
---

# rostlabs/rost-286m-sft

286M parameter nanochat GPT (sft stage), research tag `normalized-d12-004b`, checkpoint step 000600.

Part of the **rost model zoo** — the full set of trained variants behind the rost
research series, published for reproducibility. This is a research model, not a
product.

| | |
|---|---|
| architecture | nanochat GPT, 12 layers, 768 embed, 2048 context |
| tokenizer | included under `tokenizer/` (32,768 vocab) |
| training mixture | Romanian SFT mixture (OpenLLM-Ro datasets) with input diacritic augmentation |
| stage | sft |
| val bpb (own split) | 0.49526 |

The best Romanian conversational model of the rost small series (research tag `normalized-d12-004b`). Input-side diacritic augmentation makes it robust to unaccented Romanian as actually typed (`cum te cheama?`). Base: [rost-286m-base](https://huggingface.co/rostlabs/rost-286m-base).

Validation bits-per-byte is measured on this arm's **own** validation split and is
**not comparable across arms** — cross-arm comparisons in the rost write-ups are
always cross-evaluated on identical text.

## Licence

**CC-BY-NC-4.0, non-commercial**, inherited from the most restrictive component
of the training data.