metadata
language:
- or
license: apache-2.0
tags:
- ocr
- odia
- qwen2.5-vl
- fine-tuned
- vision-language
- lora
datasets:
- shantipriya/odia-ocr-merged
base_model: Qwen/Qwen2.5-VL-3B-Instruct
pipeline_tag: image-text-to-text
Odia OCR — Qwen2.5-VL-3B Fine-tuned (v2)
Fine-tuned Qwen2.5-VL-3B-Instruct for Odia-script OCR using LoRA on 145 K word-level crops from the merged Odia OCR dataset.
Training Progress 🏃
| Metric | Value |
|---|---|
| Base model | Qwen/Qwen2.5-VL-3B-Instruct |
| Dataset | shantipriya/odia-ocr-merged (145 K samples) |
| Total planned steps | 12,387 |
| Current step | 5,000 / 12,387 (40%) |
| Current epoch | 1.2 |
| Latest loss | ~4.78 |
| Saved checkpoints | 3200 · 4800 · 5000 |
| Strategy | LoRA r=64 α=128, bf16, batch=4×4=16 eff |
| Hardware | 1× H100 80 GB |
Training is ongoing. This repo holds the checkpoint-5000 weights as the latest stable snapshot.
Checkpoints
| Checkpoint | Step | Notes |
|---|---|---|
checkpoint-3200 |
3200 | Early stage |
checkpoint-4800 |
4800 | ~38% complete |
checkpoint-5000 |
5000 | Latest pushed — 40% complete |
| (final) | ~12387 | ~2 epochs |
Usage
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
from peft import PeftModel
import torch
from PIL import Image
BASE = "Qwen/Qwen2.5-VL-3B-Instruct"
ADAPTER = "shantipriya/odia-ocr-qwen-finetuned_v2"
processor = AutoProcessor.from_pretrained(BASE, trust_remote_code=True)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
BASE, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, ADAPTER)
image = Image.open("odia_word_crop.png").convert("RGB")
messages = [{"role": "user", "content": [
{"type": "image", "image": image},
{"type": "text", "text": "Extract the Odia text from this image. Return only the text."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(images=[image], text=[text], return_tensors="pt").to(model.device)
with torch.no_grad():
out = model.generate(**inputs, max_new_tokens=64, temperature=0.1, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Limitations
- Trained on word-level crops → best accuracy on individual words/short lines
- Paragraph / full-page OCR: outputs only 4–16% of text (known limitation)
- Phase 3 (mixed word + paragraph training) is planned after the current run completes
Dataset
shantipriya/odia-ocr-merged — 145 K curated Odia word crops with ground-truth labels.
License
Apache 2.0