metadata
license: apache-2.0
datasets:
- alibayram/hepsiburada_yorumlar
- aliarda/turkish-news-1.8M-tokenized
language:
- tr
base_model:
- aliarda/llama-50M-randParams
- aliarda/llama-50M-latest
This is a Domain Adaptive PreTrained llama model. Made for experimental purposes.
You can use modeling files from this GitHub repo.
- Model Size: 52,177,152
- Vocab Size: 32,768
- Context Length: 512
- Embedding Dimension: 256
- Attention Heads: 128
- KV Groups: 64
- Hidden Dimension: 2048
- Number of Layers: 20
Original pretrained model is trained on 1/4 of aliarda/turkish-news-1.8M-tokenized. This model is then trained on 80% of alibayram/hepsiburada_yorumlar with the goal of adapting to another domain.