theshivam7/whisper-tiny-indian-english

Whisper Tiny: fine-tuned for Indian English

Fine-tuned openai/whisper-tiny (39M params) on TIE_shorts, a corpus of NPTEL-style Indian-English academic lecture audio. Part of Indian-ASR-Bench, a reproducible WER benchmark for ASR on Indian English speech.

Usage

from transformers import pipeline

pipe = pipeline("automatic-speech-recognition", model="theshivam7/whisper-tiny-indian-english")
result = pipe("audio.wav")
print(result["text"])

For audio longer than 30 seconds, enable chunking:

result = pipe("long_audio.wav", chunk_length_s=30, stride_length_s=5)

Model description

A full fine-tune (all 39M parameters) of Whisper Tiny via transformers Seq2SeqTrainer, using a step-based recipe (max_steps=2000, effective batch size 32, fp16, best-checkpoint selection on validation WER). Best checkpoint: step 600/2000.

Intended uses & limitations

Evaluated on Indian-accented English academic lecture speech (TIE_shorts). Not evaluated on other English varieties or domains.

⚠️ Speaker overlap (disclosed). The training data's official split shares speakers with the test split (100% of test speakers, and 100% of test clips, are from speakers also seen in training). There is no clip-level leakage, but part of the measured gain over the pretrained baseline likely reflects speaker adaptation rather than purely accent/content adaptation. See the benchmark repo for full methodology.

⚠️ Gain is not statistically confirmed. The −2.96pp WER improvement over the pretrained baseline (below) does not survive Holm–Bonferroni correction across the 3-model fine-tuning family at 985 test clips (95% CI [−6.35, +0.13] pp, p_Holm=0.195); read it as suggestive, not conclusive.

Training and evaluation data

• Train: TIE_shorts official train split, 7,200 filtered clips (46.9h).
• Eval: TIE_shorts official test split, 985 scored clips (one excluded for an empty reference).

Training procedure

Base model	openai/whisper-tiny (39M)
Steps	2000 (best checkpoint at step 600)
Effective batch size	32
Precision	fp16
Optimizer	AdamW (transformers Seq2SeqTrainer default)
Selection	best checkpoint by validation WER

Evaluation results

Corpus WER on TIE_shorts test split, transcript_clean normalization, both models decoded through the identical HF transformers pipeline (engine-controlled comparison):

Model	Corpus WER
openai/whisper-tiny (pretrained baseline)	22.10%
theshivam7/whisper-tiny-indian-english (this model)	19.14%

Δ = −2.96 pp (95% CI [−6.35, +0.13], p_Holm=0.195, not significant after correction, see limitations above).

Framework versions

• Transformers, PyTorch, Datasets (see training environment)

Citation

Part of Indian-ASR-Bench. See the repo for full benchmark methodology, error analysis, and reproduction instructions.