ak-lm-small

Tiny pretrained Malayalam language model (75M parameters)

AK-LM-SMALL (Akshara Language Model Small) is a 75M small Malayalam GPT style Model ,Build from Scratch

Developed by EnduraSolution.

Overview

GPT-style decoder-only Transformer
75 Million parameters
Custom Malayalam SentencePiece tokenizer
Vocabulary: 24,000
Context length: 256

Training Data

IndicCorp Malayalam
Malayalam Wikipedia
Kerala-focused corpus
Small English Wikipedia subset

Intended Uses

Malayalam text generation
Text completion
Research
Fine-tuning

Limitations

This is a pretrained base model.

It is not instruction tuned.

Example

from transformers import GPT2LMHeadModel
import sentencepiece as spm

sp = spm.SentencePieceProcessor(model_file='ml_sp.model')
model = GPT2LMHeadModel.from_pretrained('endurasolution/ak-lm-small')

License

Apache-2.0

Made with ❤️ by EnduraSolution