kirandevraj/whisper-medium-supreme-court-hybrid

whisper-medium · Supreme Court hearings (hybrid dataset)

A full fine-tune of openai/whisper-medium (all 763M parameters) on the kirandevraj/supreme-court-hearings-asr dataset — 46.9 h of Indian Supreme Court hearing audio, sentence-aligned by a best-of-both (MMS-CTC + Whisper) forced-alignment pipeline.

Results

Whisper BasicTextNormalizer applied to both sides; WER/CER via jiwer. Trained on the by-case train split (24,422 clips); both eval sets are held out (the gold hearing is never trained on). Best checkpoint selected by validation WER.

Eval set	Model	WER	CER
Held-out test (3,799)	whisper-medium · zero-shot	12.6	7.2
Held-out test (3,799)	whisper-medium · fine-tuned (this)	9.2	5.2
Human-verified gold (109)	whisper-medium · zero-shot	25.0	18.1
Human-verified gold (109)	whisper-medium · fine-tuned (this)	14.6	10.3

Validation WER during training reached 9.53 (CER 5.99). Fine-tuning cuts test WER ~27% relative and gold WER ~42% relative over zero-shot whisper-medium.

Training

• Base: openai/whisper-medium (769M), full fine-tune (no LoRA)
• Optimizer: AdamW, lr 1e-5, 50-step warmup, bf16, gradient checkpointing
• Batch: effective 16 (per-device 8 × grad-accum 2)
• Steps: 3000 (~2 epochs); decoding forced to English

Usage

import torch
from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="kirandevraj/whisper-medium-supreme-court-hybrid",
    torch_dtype=torch.float16,
    device=0,  # GPU; use -1 for CPU
)
out = asr("hearing_clip.wav", generate_kwargs={"language": "en", "task": "transcribe"})
print(out["text"])

Lower-level (processor + model):

import torch, soundfile as sf
from transformers import WhisperForConditionalGeneration, WhisperProcessor

repo = "kirandevraj/whisper-medium-supreme-court-hybrid"
proc = WhisperProcessor.from_pretrained(repo)
model = WhisperForConditionalGeneration.from_pretrained(repo, torch_dtype=torch.float16).to("cuda").eval()

audio, sr = sf.read("hearing_clip.wav")          # 16 kHz mono
feats = proc(audio, sampling_rate=16000, return_tensors="pt").input_features.to("cuda", torch.float16)
ids = model.generate(feats, language="en", task="transcribe")
print(proc.batch_decode(ids, skip_special_tokens=True)[0])

Notes

• Audio must be 16 kHz mono; clips ≤ 30 s (the Whisper encoder window).
• Domain: Indian Supreme Court oral hearings — English with occasional romanized Hindi, legal vocabulary, case citations, and speaker names.