metadata
license: apache-2.0
datasets:
- alibayram/tr-books
language:
- tr
🧠 LLaMA-50M Turkish Books
Model Summary
| Property | Value |
|---|---|
| Architecture | LLaMA (decoder-only transformer) |
| Parameters | ~50M |
| Vocab size | 32,768 |
| Embedding dim | 256 |
| Hidden dim | 2048 |
| Layers | 20 |
| Attention heads | 128 |
| KV groups | 64 |
| Context length | 256 |
| Tokenizer | turkish_tokenizer (alibayram) |
| Dataset | alibayram/tr_books |
| Tokens seen | 372,679,971 |
| Epochs | 2 |
| Batch size | 64 |
| Language | Turkish 🇹🇷 |
Model Description
llama-50m-turkish-books is a 50 million parameter Turkish language model trained from scratch on the alibayram/tr_books dataset.
It was developed as a lightweight experimental model to explore Turkish-specific tokenization and morphology-aware pretraining using the custom turkish_tokenizer.
The model follows the LLaMA-style causal transformer architecture and was trained with a context length of 256 tokens over ~372M tokens in total.
Training Environment
Hardware: NVIDIA B200 GPU ~7 hours