whisper-medium-mizonal3-E1-lus-v2026.06

This model is a fine-tuned version of openai/whisper-medium on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

• Wer: 21.6845
• Cer: 6.6429
• Real Time Factor: 0.0579

Quick Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.91	0.6204	0.6075	191.75	122.09	1.49e-04	6.02
500	1.82	0.7347	0.7841	61.61	29.12	2.99e-04	5.98
750	2.73	0.5891	0.7306	50.03	22.44	2.56e-04	4.86
1000	3.64	0.3789	0.6051	32.77	13.89	2.12e-04	3.08
1250	4.55	0.2269	0.5719	28.95	11.41	1.68e-04	2.71
1500	5.46	0.1241	0.5292	38.97	23.80	1.24e-04	1.89
1750	6.36	0.0627	0.5197	23.02	8.40	7.96e-05	1.37
2000	7.27	0.0208	0.4816	19.56	6.01	3.55e-05	0.73