trocr-omr-handwritten

Fine-tuned microsoft/trocr-base-handwritten for reading handwritten fields (course code, registration number, roll number, marks obtained) from OMR answer sheets.

Evaluation results (inference set, all categories included)

Category	# Examples	CER	Exact Match Accuracy
course_code	285	0.0012	0.9930
marks_obtained	148	0.0034	0.9932
registration_no	332	0.0055	0.9548
roll_no	287	0.0199	0.9303
Overall	1052	0.0083	0.9639

CER = Character Error Rate (lower is better). Exact Match Accuracy is the fraction of predictions that match the ground truth exactly.

Usage

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
 
processor = TrOCRProcessor.from_pretrained("anjali214/trocr-omr-handwritten")
model = VisionEncoderDecoderModel.from_pretrained("anjali214/trocr-omr-handwritten")
 
image = Image.open("your_field_crop.png").convert("RGB")
pixel_values = processor(image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)

Training data

Fine-tuned on a private OMR answer sheet dataset (course codes, registration numbers, roll numbers, and marks obtained), split 70/15/15 into train/validation/inference.