README.md

---
license: gemma
base_model: unsloth/gemma-4-E4B-it
language:
- hi
- en
tags:
- hindi
- indic
- gemma
- unsloth
- text-generation
datasets:
- ai4bharat/indic-instruct-data-v0.1
pipeline_tag: text-generation
library_name: transformers
---

# 🇮🇳 Gemma-4-E4B-Hindi-Instruct (16-bit)

A Hindi instruction-tuned fine-tune of **Gemma 4 E4B**. This is the merged 16-bit model for use with 🤗 Transformers / vLLM / further fine-tuning.

For local CPU/edge use, see the **GGUF** build.

> Part of my **Hindi LLM Series** — small, openly-documented Indic models that actually follow instructions in Hindi and run on your own machine.

---

## Usage (Transformers)

```python
from transformers import AutoModelForCausalLM, AutoProcessor
import torch

model_id = "pankajpandey-dev/gemma-4-e4b-hindi-instruct"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
proc  = AutoProcessor.from_pretrained(model_id)

msgs = [{"role": "user", "content": [{"type": "text", "text": "मशीन लर्निंग को आसान शब्दों में समझाओ।"}]}]
inputs = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
                                  return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, use_cache=True)
print(proc.decode(out[0], skip_special_tokens=True))
```

---

## Example outputs

**Prompt:** भारत के बारे में एक रोचक तथ्य बताओ।
> भारत दुनिया में सबसे अधिक भाषाओं वाले देशों में से एक है — 22 आधिकारिक भाषाएँ और 1,000 से अधिक बोलियाँ। हिंदी एक इंडो-आर्यन भाषा है, जबकि तमिल एक द्रविड़ भाषा है।

---

## Training details

| | |
|---|---|
| Base model | `unsloth/gemma-4-E4B-it` |
| Method | LoRA (r=16, α=16), response-only loss |
| Framework | [Unsloth](https://github.com/unslothai/unsloth) |
| Data | ~10k Hindi instruction pairs (AI4Bharat indic-instruct: anudesh + dolly, hi splits) |
| Epochs | 2 |
| LR / schedule | 1e-4, cosine |
| Precision | bf16 (4-bit QLoRA base) |
| Hardware | Single NVIDIA L4 (24 GB) |
| Final train loss | ~0.29 |

Trained text-only (vision layers frozen), single-BOS chat template to avoid double-BOS corruption.

---

## Related repos

- GGUF (Q4/Q5/Q8): [`pankajpandey-dev/gemma-4-e4b-hindi-instruct-GGUF`](https://huggingface.co/pankajpandey-dev/gemma-4-e4b-hindi-instruct-GGUF)
- LoRA adapter: [`pankajpandey-dev/gemma-4-e4b-hindi-instruct-lora`](https://huggingface.co/pankajpandey-dev/gemma-4-e4b-hindi-instruct-lora)

---

## Provenance & license (please read)

Mixed-license lineage — review all before redistribution or commercial use:

- **Weights** derive from **Gemma 4**, under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).
- **Data** from [AI4Bharat indic-instruct-data-v0.1](https://huggingface.co/datasets/ai4bharat/indic-instruct-data-v0.1):
  - **Dolly** split — from `databricks-dolly-15k`, **CC-BY-SA-3.0**.
  - **Anudesh** split — responses from **Llama-2-70B**, so the **Llama 2 Community License** applies.

Raw training data is not redistributed here. You are responsible for complying with the Gemma, Llama 2, and CC-BY-SA terms.

---

## Limitations

- ~8B-class model: strong Hindi fluency, but can hallucinate facts and occasionally repeat phrasing on long open-ended generation.
- Tuned for single-turn Hindi instructions; long multi-turn chat is not the focus.
- Not safety-aligned for production.

## Acknowledgements

Base model by Google (Gemma 4). Data by AI4Bharat. Fine-tuning with Unsloth. 🙏