Add README.md

README.md

@@ -0,0 +1,116 @@
+---
+license: cc-by-sa-4.0
+base_model: google/gemma-4-E4B-it
+library_name: peft
+tags:
+  - lora
+  - gemma-4
+  - multilingual
+  - low-resource-languages
+  - endangered-languages
+language:
+  - multilingual
+datasets:
+  - facebook/flores
+  - shiyue/chr_en
+---
+
+# LinguaForge — Gemma 4 E4B LoRA across 204 languages
+
+A single ~170 MB LoRA adapter that shifts Google DeepMind's
+[`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it)
+toward every language in **FLORES-200** (Meta NLLB Team, *No Language Left
+Behind*, Nature 2024) plus Cherokee depth from the **ChrEn** corpus
+(Zhang, Frey & Bansal, EMNLP 2020). Cherokee is *not* in FLORES-200.
+
+Trained as part of the **LinguaForge / 古韵 GuYun** submission to the
+Gemma 4 Hackathon (`AI for Good — endangered language preservation`).
+
+## Training summary (Kaggle T4, ~5 h 9 min)
+
+| | |
+|---|---|
+| Base model | `unsloth/gemma-4-e4b-it-unsloth-bnb-4bit` (4-bit NF4) |
+| Trainable params | 42,401,792 / 8,038,558,240 (0.53%) |
+| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
+| Rank / alpha / dropout | 16 / 32 / 0.05 |
+| Total chat samples | 33,480 (alt. `en → target` / `target → en`) |
+| Languages covered | **203 FLORES-200 languages + Cherokee from ChrEn = 204** |
+| Continents | 6 (Africa, Asia, Europe, Pacific, South America, Diaspora) + N. America |
+| Optimizer steps | 8,370 (1 epoch, batch 2 × grad-accum 2) |
+| Reproducer | Kaggle kernel [`dongwei666/linguaforge-auto`](https://www.kaggle.com/code/dongwei666/linguaforge-auto) |
+
+## Held-out evaluation (FLORES-200 devtest + ChrEn seed=99)
+
+Numbers from Kaggle kernel
+[`dongwei666/linguaforge-eval`](https://www.kaggle.com/code/dongwei666/linguaforge-eval),
+50 unseen sentences per language, greedy decoding, `sacrebleu` corpus-level
+metrics.
+
+| Language | base BLEU | +LoRA BLEU | Δ BLEU | base chrF | +LoRA chrF | Δ chrF |
+|---|---:|---:|---:|---:|---:|---:|
+| Cherokee (`chr_Cher`) | 0.04 | **0.45** | +0.41 | 2.30 | **7.87** | **+5.56** (3.4×) |
+| Tibetan (`bod_Tibt`)  | 0.12 | **0.21** | +0.09 | 19.14 | **27.05** | **+7.91** |
+| Welsh (`cym_Latn`)    | 3.90 | **6.13** | **+2.23** | 31.11 | 31.21 | +0.10 |
+| Quechua (`quy_Latn`)  | 1.02 | **1.93** | +0.91 | 19.94 | **22.49** | +2.55 |
+| Māori (`mri_Latn`)    | 3.64 | **4.16** | +0.52 | 28.48 | 27.58 | −0.90 |
+| Yoruba (`yor_Latn`)   | 2.54 | 1.12 | −1.42 | 21.65 | 11.10 | −10.55 ⚠ |
+| **Mean (6 langs)**    | **1.88** | **2.33** | **+0.45** | **20.44** | **21.22** | **+0.78** |
+
+Honest read: the LoRA's biggest wins are on **languages whose scripts the
+base model could barely write** (Cherokee chrF 3.4×, Tibetan chrF +7.91).
+Welsh shows the largest BLEU jump (+2.23) — the adapter strips a
+`**Welsh Translation:**` boilerplate prefix from the base model. Yoruba
+regressed into a repetition loop; reported transparently. With more
+samples per language or per-community LoRAs, that regression should
+resolve.
+
+## Usage
+
+```python
+from unsloth import FastLanguageModel
+from unsloth.chat_templates import get_chat_template
+import torch
+
+model, tok = FastLanguageModel.from_pretrained(
+    model_name="zcgf111/linguaforge-gemma4-204lang-lora",
+    max_seq_length=2048,
+    load_in_4bit=True,
+)
+tok = get_chat_template(tok, chat_template="gemma")
+FastLanguageModel.for_inference(model)
+
+msgs = [
+    {"role": "system", "content": "You are LinguaForge, a multilingual tutor for endangered and low-resource languages."},
+    {"role": "user",   "content": "Translate this English sentence into Maori (Polynesian, Pacific):\n\nHello, my name is Sarah."},
+]
+text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+inputs = tok.tokenizer(text, return_tensors="pt").to(model.device)
+with torch.inference_mode():
+    out = model.generate(**inputs, max_new_tokens=128, do_sample=False,
+                         pad_token_id=tok.tokenizer.eos_token_id)
+print(tok.tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+## Citations
+
+```bibtex
+@article{nllb2024,
+  title={Scaling neural machine translation to 200 languages},
+  author={{NLLB Team} and Costa-juss{\`a}, Marta R. and others},
+  journal={Nature},
+  year={2024},
+  doi={10.1038/s41586-024-07335-x}
+}
+@inproceedings{zhang-etal-2020-chren,
+  title={{ChrEn}: {Cherokee-English} Machine Translation for Endangered Language Revitalization},
+  author={Zhang, Shiyue and Frey, Benjamin and Bansal, Mohit},
+  booktitle={EMNLP},
+  year={2020}
+}
+```
+
+## License
+
+CC-BY-SA 4.0, matching the FLORES-200 license. ChrEn is released
+under CC-BY-SA 4.0 by its authors.