Update README.md
Browse files
README.md
CHANGED
|
@@ -20,7 +20,7 @@ language:
|
|
| 20 |
| **Attention heads** | 128 |
|
| 21 |
| **KV groups** | 64 |
|
| 22 |
| **Context length** | 256 |
|
| 23 |
-
| **Tokenizer** | [turkish_tokenizer (alibayram)](https://github.com/
|
| 24 |
| **Dataset** | [alibayram/tr_books](https://huggingface.co/datasets/alibayram/tr_books) |
|
| 25 |
| **Tokens seen** | 372,679,971 |
|
| 26 |
| **Epochs** | 2 |
|
|
@@ -32,7 +32,7 @@ language:
|
|
| 32 |
## Model Description
|
| 33 |
|
| 34 |
`llama-50m-turkish-books` is a **50 million parameter Turkish language model** trained from scratch on the `alibayram/tr_books` dataset.
|
| 35 |
-
It was developed as a lightweight experimental model to explore **Turkish-specific tokenization and morphology-aware pretraining** using the custom [turkish_tokenizer](https://github.com/
|
| 36 |
|
| 37 |
The model follows the **LLaMA-style causal transformer** architecture and was trained with a context length of 256 tokens over ~372M tokens in total.
|
| 38 |
|
|
|
|
| 20 |
| **Attention heads** | 128 |
|
| 21 |
| **KV groups** | 64 |
|
| 22 |
| **Context length** | 256 |
|
| 23 |
+
| **Tokenizer** | [turkish_tokenizer (alibayram)](https://github.com/malibayram/turkish-tokenizer) |
|
| 24 |
| **Dataset** | [alibayram/tr_books](https://huggingface.co/datasets/alibayram/tr_books) |
|
| 25 |
| **Tokens seen** | 372,679,971 |
|
| 26 |
| **Epochs** | 2 |
|
|
|
|
| 32 |
## Model Description
|
| 33 |
|
| 34 |
`llama-50m-turkish-books` is a **50 million parameter Turkish language model** trained from scratch on the `alibayram/tr_books` dataset.
|
| 35 |
+
It was developed as a lightweight experimental model to explore **Turkish-specific tokenization and morphology-aware pretraining** using the custom [turkish_tokenizer](https://github.com/malibayram/turkish-tokenizer).
|
| 36 |
|
| 37 |
The model follows the **LLaMA-style causal transformer** architecture and was trained with a context length of 256 tokens over ~372M tokens in total.
|
| 38 |
|