Update README: training progress at step 5000/12387

README.md

@@ -1,377 +1,88 @@
 ---
-base_model: Qwen/Qwen2.5-VL-3B-Instruct
-library_name: peft
-pipeline_tag: image-to-text
 language:
 - or
+license: apache-2.0
 tags:
-- odia
 - ocr
-- vision-language
-- lora
+- odia
 - qwen2.5-vl
 - fine-tuned
-- transformers
-- peft
-license: apache-2.0
+- vision-language
+- lora
 datasets:
 - shantipriya/odia-ocr-merged
+base_model: Qwen/Qwen2.5-VL-3B-Instruct
+pipeline_tag: image-text-to-text
 ---
 
 # Odia OCR — Qwen2.5-VL-3B Fine-tuned (v2)
 
-A **LoRA fine-tuned** vision-language model for **Odia script Optical Character Recognition (OCR)**.  
-Built on top of `Qwen/Qwen2.5-VL-3B-Instruct`, fine-tuned on 145K+ Odia word images.
-
----
+Fine-tuned **Qwen2.5-VL-3B-Instruct** for Odia-script OCR using LoRA on 145 K word-level crops from the merged Odia OCR dataset.
 
-## Model Summary
+## Training Progress 🏃
 
-| Property | Value |
-|---|---|
+| Metric | Value |
+|--------|-------|
 | Base model | `Qwen/Qwen2.5-VL-3B-Instruct` |
-| Fine-tuning method | LoRA (r=128, alpha=256) |
-| Training dataset | `shantipriya/odia-ocr-merged` (145K+ samples) |
-| Training hardware | NVIDIA H100 80 GB |
-| Training framework | HuggingFace Transformers + PEFT |
-| Language | Odia (ଓଡ଼ିଆ) |
-| Task | OCR — image to Odia text |
-| License | Apache 2.0 |
-
----
-
-## Evaluation Results
-
-Evaluated on the held-out test split of `shantipriya/odia-ocr-merged`.
-
-### Checkpoint Progress
-
-| Checkpoint | Step | % Complete | Exact Match | Mean CER |
-|---|---|---|---|---|
-| checkpoint-3600 | 3600 | 29% | 68.2% | 9.60% |
-| checkpoint-4400 | 4400 | 36% | **68.0%** | **8.33%** |
-
-> Training is ongoing (12,387 total steps). Results will be updated as training progresses.
-
-### Error Analysis (checkpoint-4400, 50 test samples)
-
-- **34/50 exact matches (68.0%)**
-- **Mean CER: 8.33%** (improved from 9.60% at step 3600)
-- Errors concentrated on visually similar Odia diacritic variants (e.g. `ଵ` vs `ବ`, `ଧ୍ଵ` vs `ଧ୍ୱ`)
-- Numeric-only samples (e.g. `୫୨`) are handled perfectly
-
----
-
-## Model Inference Results
-
-![Inference Results](assets/inference_paragraph.png)
-
-Sample predictions from checkpoint-4400 on 10 test images:
-
-| # | Image | Ground Truth | Predicted | Match | CER |
-|---|---|---|---|---|---|
-| 1 | ![](assets/sample_1.png) | ଚଳେଇବାପାଇଁ | ଚଳେଇବାପାଇଁ | ✅ | 0.0% |
-| 2 | ![](assets/sample_2.png) | ପୁନର୍ନଭା | ପୁନର୍ନଭା | ✅ | 0.0% |
-| 3 | ![](assets/sample_3.png) | ଟ୍ରଷ୍ଟର | ଟୁଷ୍ଟର | ⚠️ | 23.1% |
-| 4 | ![](assets/sample_4.png) | ସୀମାର | ସୀମାର | ✅ | 0.0% |
-| 5 | ![](assets/sample_5.png) | ଟ୍ୟୁବରକୁଲୋସିସ | ଟ୍ୟୁବଲକୁଲୋସିସ | ⚠️ | 7.7% |
-| 6 | ![](assets/sample_6.png) | ଟ୍ରାନ୍ସଭର୍ସସ | ଟ୍ରାନ୍ସଭର୍ସ | ⚠️ | 4.4% |
-| 7 | ![](assets/sample_7.png) | ପ୍ଲାଜ୍ମାରେ | ପ୍ଲାଜ୍ମାରେ | ✅ | 0.0% |
-| 8 | ![](assets/sample_8.png) | ଖଣ୍ଡଗିରିରେ | ଖଣ୍ଡଗିରିରେ | ✅ | 0.0% |
-| 9 | ![](assets/sample_9.png) | ହାଇପରଆନିମିଆ | ହାଇପରଆନିମିଆ | ✅ | 0.0% |
-| 10 | ![](assets/sample_10.png) | ହୋଇଯାନ୍ତି | ହୋଇଯାଆନ୍ତି | ⚠️ | 5.3% |
-
-**10-sample summary:** 7/10 exact match, mean CER 4.0%
-
----
+| Dataset | `shantipriya/odia-ocr-merged` (145 K samples) |
+| Total planned steps | 12,387 |
+| **Current step** | **5,000 / 12,387 (40%)** |
+| Current epoch | 1.2 |
+| Latest loss | ~4.78 |
+| Saved checkpoints | 3200 · 4800 · **5000** |
+| Strategy | LoRA r=64 α=128, bf16, batch=4×4=16 eff |
+| Hardware | 1× H100 80 GB |
+
+> Training is ongoing. This repo holds the **checkpoint-5000** weights as the latest stable snapshot.
+
+## Checkpoints
+
+| Checkpoint | Step | Notes |
+|------------|------|-------|
+| `checkpoint-3200` | 3200 | Early stage |
+| `checkpoint-4800` | 4800 | ~38% complete |
+| `checkpoint-5000` | 5000 | **Latest pushed** — 40% complete |
+| *(final)* | ~12387 | ~2 epochs |
 
 ## Usage
 
-### Quick Inference
-
 ```python
-import torch
-from PIL import Image
 from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
 from peft import PeftModel
-from qwen_vl_utils import process_vision_info
+import torch
+from PIL import Image
 
-BASE_MODEL = "Qwen/Qwen2.5-VL-3B-Instruct"
-ADAPTER    = "shantipriya/odia-ocr-qwen-finetuned_v2"
+BASE   = "Qwen/Qwen2.5-VL-3B-Instruct"
+ADAPTER = "shantipriya/odia-ocr-qwen-finetuned_v2"
 
-processor = AutoProcessor.from_pretrained(BASE_MODEL, trust_remote_code=True)
+processor = AutoProcessor.from_pretrained(BASE, trust_remote_code=True)
 model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
-    BASE_MODEL, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
+    BASE, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
 )
 model = PeftModel.from_pretrained(model, ADAPTER)
-model.eval()
 
-def ocr(image_path: str) -> str:
-    image = Image.open(image_path).convert("RGB")
-    messages = [{"role": "user", "content": [
-        {"type": "image", "image": image},
-        {"type": "text",  "text": "Extract the Odia text from this image. Return only the text."}
-    ]}]
-    text_input = processor.apply_chat_template(
-        messages, tokenize=False, add_generation_prompt=True
-    )
-    image_inputs, _ = process_vision_info(messages)
-    inputs = processor(
-        text=[text_input], images=image_inputs, return_tensors="pt"
-    ).to(model.device)
-    with torch.no_grad():
-        out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
-    return processor.decode(
-        out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True
-    ).strip()
-
-print(ocr("your_odia_image.png"))
+image = Image.open("odia_word_crop.png").convert("RGB")
+messages = [{"role": "user", "content": [
+    {"type": "image", "image": image},
+    {"type": "text",  "text":  "Extract the Odia text from this image. Return only the text."},
+]}]
+text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = processor(images=[image], text=[text], return_tensors="pt").to(model.device)
+with torch.no_grad():
+    out = model.generate(**inputs, max_new_tokens=64, temperature=0.1, do_sample=False)
+print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
 ```
 
-### Install Dependencies
-
-```bash
-pip install transformers peft accelerate qwen-vl-utils torch pillow
-```
-
----
-
-## Training Details
-
-### Dataset
-
-- **Source:** [shantipriya/odia-ocr-merged](https://huggingface.co/datasets/shantipriya/odia-ocr-merged)
-- **Size:** 145,000+ word-level OCR samples
-- **Format:** image → Odia Unicode text pairs
-- **Word length:** 3–20 characters per sample
-
-### LoRA Configuration
-
-```python
-LoraConfig(
-    r=128,
-    lora_alpha=256,
-    target_modules=[
-        "q_proj", "k_proj", "v_proj", "o_proj",
-        "gate_proj", "up_proj", "down_proj"
-    ],
-    lora_dropout=0.05,
-    bias="none",
-    task_type="CAUSAL_LM",
-)
-```
-
-### Training Hyperparameters
-
-| Parameter | Value |
-|---|---|
-| Per-device batch size | 4 |
-| Gradient accumulation steps | 4 (effective batch = 16) |
-| Learning rate | 2e-4 |
-| LR scheduler | Cosine |
-| Warmup steps | 100 |
-| Total steps | 12,387 |
-| Optimizer | AdamW (bf16) |
-| Hardware | NVIDIA H100 80 GB |
-
----
-
 ## Limitations
 
-- Optimised for **word-level OCR** (short text, 3–20 chars)
-- May struggle with very long text sequences or full document OCR
-- Some confusion between visually similar Odia diacritics (conjunct consonants)
-- Performance improves as training continues toward 12,387 total steps
-
----
+- Trained on **word-level** crops → best accuracy on individual words/short lines
+- Paragraph / full-page OCR: outputs only 4–16% of text (known limitation)
+- **Phase 3** (mixed word + paragraph training) is planned after the current run completes
 
-## Citation
+## Dataset
 
-```bibtex
-@misc{odia-ocr-qwen-v2-2026,
-  title  = {Odia OCR Fine-tuned Qwen2.5-VL-3B (v2)},
-  author = {Shantipriya Parida},
-  year   = {2026},
-  url    = {https://huggingface.co/shantipriya/odia-ocr-qwen-finetuned_v2}
-}
-```
-
----
+`shantipriya/odia-ocr-merged` — 145 K curated Odia word crops with ground-truth labels.
 
-## Related Resources
-
-- Base model: [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct)
-- Dataset: [shantipriya/odia-ocr-merged](https://huggingface.co/datasets/shantipriya/odia-ocr-merged)
-- OdiaGenAI copy: [OdiaGenAIOCR/odia-ocr-qwen-finetuned](https://huggingface.co/OdiaGenAIOCR/odia-ocr-qwen-finetuned)
-
----
-
-### Framework Versions
-
-- PEFT 0.18.1
-- Transformers >= 4.48.0
-- PyTorch >= 2.5.0 (CUDA 12.4)
-
-## 📄 Paragraph OCR Samples
-
-Samples from [OdiaGenAIOCR/Odia-lipi-ocr-data](https://huggingface.co/datasets/OdiaGenAIOCR/Odia-lipi-ocr-data).
-Each shows the original paragraph image, the ground truth, and the model's extracted text.
-
-### ⚠️ Current Limitation — Paragraph OCR
-
-The current fine-tune (checkpoint-4800) was trained **exclusively on word-level crops**
-from `shantipriya/odia-ocr-merged` (~145K samples, each containing 1–3 Odia words).
-This means:
-
-| Issue | Cause |
-|---|---|
-| Model outputs short fragments | Training distribution = short strings → model stops early |
-| Hallucinated text | Full-page images are out-of-distribution; model generates plausible-but-wrong Odia |
-| Even tiled strips fail | A 400 px strip (~5 text lines) is still OOD for a word-crop OCR model |
-
-**Fix (Phase 3):** `train_mixed_para_word.py` mixes 20 % paragraph samples from
-`OdiaGenAIOCR/Odia-lipi-ocr-data` with 80 % word samples and warm-starts from
-checkpoint-4800. This teaches the model to recognise multi-line input as a cue to
-generate longer output.
-
----
-
-### Sample 1 — index 14 (1477×2126 px)
-
-<img src="https://huggingface.co/shantipriya/odia-ocr-qwen-finetuned_v2/resolve/main/assets/para_sample_1.png" width="380" alt="Paragraph sample 1">
-
-**Ground truth** (excerpt, 1203 chars):
-```
-ବାଳକକ୍ଷା
-ଗୋଶୃଙ୍ଗ ନଖ କେଶୈସ୍ତୁ ଧୂପତ୍ୟେଦ୍ୱାଳକଂ ତତଃ।
-ମଗ୍ନ ସ୍ନାନାୟକଂ ସବଂ ପ୍ରଥମେ ଧ୍ରୁବମେଣ ବୈ॥
-
-ନବମ ଦିବସ ବାଇ ରକ୍ଷା
-ନବମେ ଦିବସେ କାଳୀ ମେତା ଗୃଦ୍ଘାତ ବୈ ଶିଶୁ।
-ତଇଷ୍ଟ୍ରା ୱାସନୋଦ୍ଦେ ଶଃ ସ୍ୱମୁଷ୍ଟି ଦ୍ବସ୍ତ ଖାଦନ॥
-ବଗ୍ଧ ଚନ୍ଦନ କୁଷ୍ଠୋସ୍ଥା ସର୍ବପି ହ୍ରଦୟେ ଲେପଭୂ।
-ନଖ ବାନର ଛେମ୍ମାଦ ଧୂପ ଯେଓ ସ।
-
-ଦଶମ ଦିବସ ବାଇ ରକ୍ଷା
-ଦଶମେ ଦିବସେ ନାମ୍ନୀ ଶ୍ୱେଦନା ନମତେ ଶଂ।
-ଉତ୍ତିଷ୍ଠ ଜ୍ୱର କହନଂ ରୌଦ୍ର ବେଦନଂ ମୁଷ୍ଟି କନ୍ଧନଂ॥
-କୁଷ୍ଠୋଗ ସକ୍ତ ସିଦ୍ଧାର୍ଥି ଲଖେଳ କମ୍ବେନ ଧୂ...
-```
-
-**Model output** (text coverage ≈ 9% with strip tiling — improving in Phase 3):
-```
-ତଳବେ
-ନଦେଲାସିଛା
-ବୁଝାଏଇଥିଲା
-ଚେଳମିଠାଚିଲକିହେବାରେପଡ଼ିଲା।ଯାଉଛିନେତାଙ୍କନାଥିବାର
-ରାବୁତିଦେଓ
-ବାଣିଜ୍ଯକୁଣ୍ଡମାନଙ୍କାଳକୁରାଜାତାଃ,ସଂରକ୍ଷଣ
-```
-
----
-
-### Sample 2 — index 3 (1418×2186 px)
-
-<img src="https://huggingface.co/shantipriya/odia-ocr-qwen-finetuned_v2/resolve/main/assets/para_sample_2.png" width="380" alt="Paragraph sample 2">
-
-**Ground truth** (excerpt, 1099 chars):
-```
-ସୂଚିପତ୍ର​
-
-ବିଷୟ ପୃଷ୍ଠଙ୍କ
-୧ । ମାତୃସ୍ତବ 	୧
-୨ । ଶରଣ 	୪
-୩ । ଗୋଟିଏ ପ୍ରାର୍ଥନା ୬
-୪ । କୃଟକ 	୮
-୫ । ମୌନୀ 	୧୦
-୬ । ବଞ୍ଚି ମୁଁ ରହିବ ଆଉ କେଉଁ
-ସୁଖ ଆଶେ ?     ୧୧
-୭ । କପୋତ କପୋତୀ ୧୩
-୮ । ବୃକ୍ଷବଟିକା 	୧୪
-୯ । ଜାପାନ ରୁଷ 	୧୬
-୧୦ । ମୋ—ମୋହନବଂଶୀ ୧୮
-୧୧ । ଯୁଗ୍ମ କୁସୁମ 	୨୦
-୧୨ । ମୁଁ 		୨୨
-୧୩ । କ୍ଷଣିକ 	୨୪
-୧୪ । ମନ ଉଚ୍ଚାଟନ ୨୫
-୧୫ । ଭଣ୍ତ ସନ୍ନ୍ଯାସୀ ୨୬
-୧୬ । ସୁଖ ଦୁଃଖର ପରିଣାମ ୨୮
-୧୭ । ମେଲାଣି 	୨୯
-୧୮ । କବିତା ସାନ୍ତୁନା ୩୦
-୧୯ । ସ୍ବପ୍ନ ଦେବୀ 	୩୧
-୨୦ ।...
-```
-
-**Model output** (text coverage ≈ 4% with strip tiling — improving in Phase 3):
-```
-ସଂପତ୍ର
-ବଚ୍ଚୁ
-କାମେସ
-ମେ-ମେଚେଦଶ
-ବୁଝିଛାଏ
-୨୩ମୂର୍ତ୍ତି
-```
-
----
-
-### Sample 3 — index 35 (945×1654 px)
-
-<img src="https://huggingface.co/shantipriya/odia-ocr-qwen-finetuned_v2/resolve/main/assets/para_sample_3.png" width="380" alt="Paragraph sample 3">
-
-**Ground truth** (excerpt, 569 chars):
-```
--BABU GIRISH CHANDRA BASU. M. A.
-» BYOMKESH CHAKRAVERTY. M. A.
-
-Dear Sirs,
-• With sentiments of deep respect and affection
-I beg to inscribe this treatise on Agriculture to
-you in appreciation of the noble zeal which ani-
-mated you to acquire knowledge of the useful
-science of Agriculture in spite of the perils of
-the sea and thereby to render important service
-to this country by bringing your sci...
-```
-
-**Model output** (text coverage ≈ 11% with strip tiling — improving in Phase 3):
-```
-ବାରୁସ୍ଥଳିଚାର୍ଯ୍ୟ
-ବାବୁଗିବିଶଚନ୍ଦ୍ରବାସୁ.ମାଆ
-ଯୁନାିଗ୍ରାଫି
-ଯୁସ୍,
-ସୁଜନି
-```
-
----
-
-### Sample 4 — index 31 (1024×1654 px)
-
-<img src="https://huggingface.co/shantipriya/odia-ocr-qwen-finetuned_v2/resolve/main/assets/para_sample_4.png" width="380" alt="Paragraph sample 4">
-
-**Ground truth** (excerpt, 821 chars):
-```
-[1]
-ଦ୍ଵିତୀୟ ପ♦ ଉତ୍ତୋଳନ ।
-( ଶୀତର ସଭାଗୃହସ୍ଥ ୟୁଷ୍ପଷ୍ଠିଭ, ଭୀମ, ନକୁଲ, ସହଦେବ ସାଈ
-ନେକ୍ସରେ ଅଧେ।ବଦନରେ ଦଣ୍ଡାମାନ )
-ଯୁଷ୍ପଷ୍ଠିର ।— ଅହୋ ! ଅଳ୍ପ କ ଦୁର୍ଯୋଗ ଉପସ୍ଥିତ ମୋହର
-ଊର୍ଦ୍ଧ୍ଵ ଜୀବନ କାନ୍ଥ କ ଶସ୍ତ୍ର ଭଗକରୁନାQ” ।
-ଅଜ ମୋହର ପ୍ରାଣ ରୁ ପ୍ରି ପୂତମ ସୁ ଭଦ୍ରାକୁମ→
-ଅଉ ଏ ସଂସାରେ ନାହାଈ । ମୁଁ ଭଣ୍ଡାଲ, ସେହ
-ଶିଶୁକ୍ଷକ ଏଡେ ଗୁ ବ୍ଲୁଭର କର୍ମ ସାଧନରେ କାଉଁ’କ
-ଜ��ୁକ୍ତ କଲ । ଅଳ ମୋଠାରୁ ନୃଶଂସ, ଋଣ୍ଡାଲ;
-ପାପିଷ୍ଠ, ନଗ୍‌ଧମ, ସ୍ଵକ୍ଷସ ଏ.ମନ୍ତ୍ରୀ ମଣ୍ଡଲରେ ନାହାଈ ।
-ଯ...
-```
-
-**Model output** (text coverage ≈ 15% with strip tiling — improving in Phase 3):
-```
-ଦୃଶ୍ୟପଟଳାଳନା
-ଶାବ୍କଚାର୍ଯ୍ୟାଦ୍ୟକାର,ଜାମ,ନକଳା,ହୁଏବାର
-ଶ୍ରିବିଜ୍ଯକୁଣ୍ଠାନ୍ତାଙ୍କୁଣ୍ଡା
-ସମାବର୍ତ୍ତେଆସାଇଏଛିଓକୁଛାନାଳୁ
-ବରୁଏଶାମୋରଚ୍ଛିଦରୁହକରିଥିଲା
-```
-
----
+## License
 
+Apache 2.0