| --- |
| license: apache-2.0 |
| datasets: |
| - alibayram/hepsiburada_yorumlar |
| - aliarda/turkish-news-1.8M-tokenized |
| language: |
| - tr |
| base_model: |
| - aliarda/llama-50M-randParams |
| - aliarda/llama-50M-latest |
| --- |
| This is a Domain Adaptive PreTrained llama model. Made for experimental purposes. |
|
|
| You can use modeling files from [this GitHub repo](https://github.com/ardafincan/LM-playground). |
|
|
| - Model Size: 52,177,152 |
| - Vocab Size: 32,768 |
| - Context Length: 512 |
| - Embedding Dimension: 256 |
| - Attention Heads: 128 |
| - KV Groups: 64 |
| - Hidden Dimension: 2048 |
| - Number of Layers: 20 |
|
|
| Original pretrained model is trained on 1/4 of aliarda/turkish-news-1.8M-tokenized. |
| This model is then trained on 80% of alibayram/hepsiburada_yorumlar with the goal of adapting to another domain. |
