---
language:
- or
license: apache-2.0
tags:
- ocr
- odia
- qwen2.5-vl
- fine-tuned
- vision-language
- lora
datasets:
- shantipriya/odia-ocr-merged
base_model: Qwen/Qwen2.5-VL-3B-Instruct
pipeline_tag: image-text-to-text
---

# Odia OCR — Qwen2.5-VL-3B Fine-tuned (v2)

Fine-tuned **Qwen2.5-VL-3B-Instruct** for Odia-script OCR using LoRA on 145 K word-level crops from the merged Odia OCR dataset.

## Training Progress 🏃

| Metric | Value |
|--------|-------|
| Base model | `Qwen/Qwen2.5-VL-3B-Instruct` |
| Dataset | `shantipriya/odia-ocr-merged` (145 K samples) |
| Total planned steps | 12,387 |
| **Current step** | **5,000 / 12,387 (40%)** |
| Current epoch | 1.2 |
| Latest loss | ~4.78 |
| Saved checkpoints | 3200 · 4800 · **5000** |
| Strategy | LoRA r=64 α=128, bf16, batch=4×4=16 eff |
| Hardware | 1× H100 80 GB |

> Training is ongoing. This repo holds the **checkpoint-5000** weights as the latest stable snapshot.

## Checkpoints

| Checkpoint | Step | Notes |
|------------|------|-------|
| `checkpoint-3200` | 3200 | Early stage |
| `checkpoint-4800` | 4800 | ~38% complete |
| `checkpoint-5000` | 5000 | **Latest pushed** — 40% complete |
| *(final)* | ~12387 | ~2 epochs |

## Usage

```python
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
from peft import PeftModel
import torch
from PIL import Image

BASE   = "Qwen/Qwen2.5-VL-3B-Instruct"
ADAPTER = "shantipriya/odia-ocr-qwen-finetuned_v2"

processor = AutoProcessor.from_pretrained(BASE, trust_remote_code=True)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    BASE, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, ADAPTER)

image = Image.open("odia_word_crop.png").convert("RGB")
messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text",  "text":  "Extract the Odia text from this image. Return only the text."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(images=[image], text=[text], return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=64, temperature=0.1, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## Limitations

- Trained on **word-level** crops → best accuracy on individual words/short lines
- Paragraph / full-page OCR: outputs only 4–16% of text (known limitation)
- **Phase 3** (mixed word + paragraph training) is planned after the current run completes

## Dataset

`shantipriya/odia-ocr-merged` — 145 K curated Odia word crops with ground-truth labels.

## License

Apache 2.0