Instructions to use diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR")
model = AutoModelForMultimodalLM.from_pretrained("diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR

SGLang

How to use diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR with Docker Model Runner:
```
docker model run hf.co/diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Sanskrit-Qwen2.5-VL-7B-Instruct-OCR

This model is a fine-tuned version of Qwen/Qwen2.5-VL-7B-Instruct on the diabolic6045/sanskrit-ocr-parallel-corpus-chat-template dataset. This data is converted from snskrt/Sanskrit_OCR_Parallel_Corpus by Sanskrit Datasets.

It achieves the following results on the evaluation set:

Loss: 0.2660
Memory/max Mem Active(gib): 20.79
Memory/max Mem Allocated(gib): 20.79
Memory/device Mem Reserved(gib): 21.46

Model description

This is a fine-tuned version of Qwen2.5-VL-7B-Instruct, specifically adapted for Sanskrit OCR (Optical Character Recognition) tasks. The model has been trained using LoRA (Low-Rank Adaptation) on a dataset of Sanskrit text images and their corresponding transcriptions.

Key Features:

Base Model: Qwen/Qwen2.5-VL-7B-Instruct (7 billion parameters)
Task: Sanskrit OCR - converting Sanskrit text images to machine-readable text
Training Method: LoRA fine-tuning with vision-language capabilities
Dataset: Sanskrit OCR Parallel Corpus with chat template formatting
Architecture: Vision-Language Model with multimodal understanding

Capabilities:

Read and transcribe Sanskrit text from images
Handle various Sanskrit scripts and fonts
Process both text and visual inputs simultaneously
Generate accurate Sanskrit text transcriptions

The model maintains the original Qwen2.5-VL's vision-language capabilities while being specialized for Sanskrit text recognition tasks.

Training and evaluation data

Training Dataset

The model was trained on the diabolic6045/sanskrit-ocr-parallel-corpus-chat-template dataset, which contains Sanskrit text images paired with their corresponding transcriptions. The dataset was converted from the original snskrt/Sanskrit_OCR_Parallel_Corpus and formatted with chat templates for vision-language training.

Evaluation Results

The model was evaluated on a test set of 314 Sanskrit text samples:

Metric	Value
Total Samples	314
Successful Samples	314
Failed Samples	0
Success Rate	100.00%
Exact Match Accuracy	1.59%
Character-level Accuracy	86.38%
Token-level Jaccard Similarity	50.44%

Key Insights:

The model successfully processes all test samples without failures
High character-level accuracy (86.38%) indicates good recognition of individual Sanskrit characters
Lower exact match accuracy (1.59%) suggests room for improvement in complete text transcription
Moderate token-level similarity (50.44%) shows reasonable semantic understanding

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 2
eval_batch_size: 2
seed: 42
distributed_type: multi-GPU
num_devices: 2
gradient_accumulation_steps: 4
total_train_batch_size: 16
total_eval_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 110
training_steps: 1105

Training results

Training Loss	Epoch	Step	Validation Loss	Mem Active(gib)	Mem Allocated(gib)	Mem Reserved(gib)
No log	0	0	3.3372	17.59	17.59	17.66
0.2428	1.0	369	0.3075	20.69	20.69	21.27
0.2057	2.0	738	0.2660	20.79	20.79	21.46

This model was trained using:

See axolotl config

axolotl version: 0.12.2

base_model: Qwen/Qwen2.5-VL-7B-Instruct
processor_type: AutoProcessor

# these 3 lines are needed for now to handle vision chat templates w images
skip_prepare_dataset: true
remove_unused_columns: false
sample_packing: false

chat_template: qwen2_vl
datasets:
  - path: sanskrit_multimodal_train.json
    type: chat_template
    field_messages: messages
dataset_prepared_path: last_run_prepared
val_set_size: 0.01
output_dir: ./outputs/out-qwen2-5-vl

adapter: lora
lora_model_dir:

sequence_len: 2048
pad_to_sequence_len: false

lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules: 'model.language_model.layers.[\d]+.(mlp|cross_attn|self_attn).(up|down|gate|q|k|v|o)_proj'

wandb_project: Sanskrit-OCR
wandb_entity:
wandb_watch:
wandb_name: qwen2-5-vl-sanskrit-ocr
wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 3
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0002

bf16: true
fp16:
tf32: true

gradient_checkpointing: true
logging_steps: 1
flash_attention: true
eager_attention:

warmup_ratio: 0.1
evals_per_epoch: 1
saves_per_epoch: 1
weight_decay: 0.0

# Automatically upload checkpoint and final model to HF
hub_model_id: diabolic6045/qwen2-5-vl-sanskrit-ocr-lora
# save_first_step: true  # uncomment this to validate checkpoint saving works with your config

Framework versions

PEFT 0.17.0
Transformers 4.55.2
Pytorch 2.7.1+cu128
Datasets 4.0.0
Tokenizers 0.21.2

Downloads last month: 46

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR

Base model

Qwen/Qwen2.5-VL-7B-Instruct

Finetuned

(1163)

this model

Quantizations

1 model

Datasets used to train diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR

Space using diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR 1

Collection including diabolic6045/Sanskrit-Qwen2.5-VL-7B-Instruct-OCR

Sanskrit LLMs

Collection

Projects I did related to make LLM better in Sanskrit • 12 items • Updated Dec 12, 2025 • 7

Evaluation results

Exact Match Accuracy on Sanskrit OCR Test Set
Sanskrit OCR Evaluation

1.590
Character-level Accuracy on Sanskrit OCR Test Set
Sanskrit OCR Evaluation

86.380
Token-level Jaccard Similarity on Sanskrit OCR Test Set
Sanskrit OCR Evaluation

50.440
Success Rate on Sanskrit OCR Test Set
Sanskrit OCR Evaluation

100.000