--- license: apache-2.0 language: - hi - en - hne tags: - hinglish - hindi - indian-languages - edge-ai - small-language-model - on-device-ai - apple-silicon - mlx - lora - qwen - qwen2 - bharat - indic-nlp - low-resource - conversational - text-generation - offline - lightweight - android base_model: Qwen/Qwen2.5-1.5B library_name: mlx pipeline_tag: text-generation datasets: - ankitdhiman/hinglish-conversations - NebulaByte/IndicInstruct - smangrul/code-py-alpaca thumbnail: https://huggingface.co/eulogik/Bharat-Tiny-LLM/resolve/main/thumbnail.png inference: false --- # 🇮🇳 Bharat-Tiny-LLM

India's first native edge AI for Hinglish & Hindi — running fully offline on ₹8,000 phones.
A 1.5B-parameter, LoRA-fine-tuned model that speaks fluent Hinglish (Romanized Hindi) and Devanagari Hindi, entirely on-device.

MLX PyPI GitHub Demo Built by eulogik License

--- ## ✨ Why Bharat-Tiny-LLM? Most Indian-language models do **one** script. Bharat-Tiny-LLM does **both** — and does it on hardware that costs less than a pair of shoes. - 🌐 **Truly bilingual** — Hinglish *and* Devanagari Hindi from a single 1.5B model. - 📱 **Edge-native** — an **880 MB 4-bit MLX build** runs offline on ₹8,000 Android phones & Apple Silicon. No API, no cloud, no internet. - 🪶 **Small but real** — 1.5B params, ~57 tok/s on a Mac Mini M4. - 💸 **$0 training cost** — fine-tuned on a Mac Mini M4, zero cloud compute. - 🆓 **Open weights** — Apache-2.0, self-hostable, no vendor lock-in. > **Built by [eulogik](https://eulogik.com)** — an India-first AI lab shipping practical, edge-deployable models. --- ## 📊 Model details | Property | Value | |----------|-------| | Base model | [Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) (multilingual, 29 languages) | | Method | LoRA — 16 layers, rank 16, alpha 32, scale 2.0, dropout 0.05 | | Trainable params | 10.55M (0.68% of 1.5B) | | Training data | **436K** cleaned Hinglish/Devanagari conversations (gold + NebulaByte + smangrul) | | Training iters | 110,000 (Mac Mini M4, batch 4, seq 768) | | Best val loss | **0.937** | | Quantization | MLX 4-bit affine, group size 64 (~880 MB) | | Inference speed | ~57 tok/s on Mac Mini M4 | | License | Apache-2.0 | --- ## 🚀 Quick start ### Python (recommended) ```bash pip install bharat-tiny-llm[mlx] # Apple Silicon pip install bharat-tiny-llm[torch] # CPU / CUDA ``` ```python from bharat_tiny_llm import chat print(chat([{"role": "user", "content": "Chai peete hain?"}])) ``` The package ships with a **canonical generation config** so output is clean out of the box: ```python temperature=0.3, top_p=0.85, repetition_penalty=1.25, no_repeat_ngram_size=3 ``` ### MLX CLI ```bash pip install mlx-lm mlx_lm chat --model eulogik/Bharat-Tiny-LLM ``` ### Transformers (fp16 fused variant) ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused") tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused") ``` > ⚠️ **Generation config matters.** The base Qwen2.5-1.5B emits garbled out-of-script > tokens at high temperature. Always use `temperature ≈ 0.3` + `repetition_penalty ≥ 1.25` > + `no_repeat_ngram_size = 3`. The `bharat-tiny-llm` package and the demo space do this for you. ### llama.cpp (GGUF — Android / Pi / CPU) ```bash # download Q4_K_M GGUF pip install llama-cpp-python ``` ```python from llama_cpp import Llama llm = Llama(model_path="bharat-tiny-llm-q4_k_m.gguf", n_ctx=1024) print(llm.create_chat_completion( messages=[{"role": "user", "content": "Chai peete hain?"}], temperature=0.3, top_p=0.85, max_tokens=256, repeat_penalty=1.25, )["choices"][0]["message"]["content"]) ``` Get the GGUF from [`eulogik/Bharat-Tiny-LLM-GGUF`](https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF). --- ## 💬 Example outputs > Generated with the 4-bit MLX build at the canonical settings above. **Q:** Chai peete hain? **A:** Haan bilkul! Chai toh humari life hai. Subah uthte hi pehli cheez jo dimag mein aati hai woh chai hi hoti hai... **Q:** Kal interview hai, nervous ho raha hoon **A:** Okay chill karo yaar! Interview mein sabse pehle confidence dikhana hota hai. Deep breath lo, apna intro clear rakho... **Q:** Biryani kaise banate hain? **A:** Biryani ek dum simple process hai, jismein kuchh ingredients hote hain aur cooking time bhi kam hota hai. Sabse important cheez hai ingredients... **Q:** Ghar baithe paise kaise kamayein? **A:** Haan, bilkul sahi sawaal hai! Ghar baithe paise kaise kamayein? Yeh toh bahut common sawaal hai aaj kal. Tension nahi lene ka, main tumhe seedha-seedha points mein batata hoon... **Q:** भारत के कुछ प्रसिद्ध त्योहार बताओ। **A:** भारत के प्रसिद्ध त्योहार हैं: दीवाली, होली, रक्षा बंधन, ईद, नवरात्रि और बहुत से और। हर त्योहार का अपना अलग महत्व है... --- ## 🗂️ Model variants | Repo | Format | Size | Use | |------|--------|------|-----| | [`eulogik/Bharat-Tiny-LLM`](https://huggingface.co/eulogik/Bharat-Tiny-LLM) | MLX 4-bit | ~880 MB | **Edge / Apple Silicon** (this repo) | | [`eulogik/Bharat-Tiny-LLM-GGUF`](https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF) | GGUF Q4_K_M | ~1.06 GB | **Cross-platform** (llama.cpp, Android, Pi, CPU) | | [`eulogik/Bharat-Tiny-LLM-fused`](https://huggingface.co/eulogik/Bharat-Tiny-LLM-fused) | PyTorch fp16 | ~3.3 GB | Server / fine-tuning base | --- ## 🧩 Limitations - 1.5B params: great for short conversational replies, weaker on long factual essays or complex reasoning. - Trained primarily on Hinglish + Hindi; other Indian languages are inherited from the Qwen2.5 base, not fine-tuned. - Quantization (4-bit) trades a little fluency for a 4× size reduction. - May occasionally repeat or drift on very long generations — keep `max_new_tokens` modest and use the n-gram guard. ## 🛣️ Roadmap - [ ] More Indic languages (Tamil, Telugu, Bengali, Marathi) - [ ] DPO / preference tuning for a quality leap - [ ] `BharatTiny-Bench` — an Indic edge-model evaluation suite - [ ] GGUF build for broader Android / Raspberry Pi deployment --- ## 🔗 Links - 🤗 Model card: https://huggingface.co/eulogik/Bharat-Tiny-LLM - 🤗 GGUF (llama.cpp): https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF - 🤗 fp16 fused: https://huggingface.co/eulogik/Bharat-Tiny-LLM-fused - 🚀 Demo space: https://huggingface.co/spaces/eulogik/Bharat-Tiny-LLM - 💻 Source code: https://github.com/eulogik/Bharat-Tiny-LLM - 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/ - 🏢 Built by [eulogik](https://eulogik.com) ## 📜 License Apache-2.0. Base weights [Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) are Apache-2.0; the LoRA adapter is Apache-2.0.