whisper-small-mizonal3-E5-lus-v2026.06

This model is a fine-tuned version of openai/whisper-small on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

• Wer: 21.0562
• Cer: 7.2973
• Real Time Factor: 0.0263

Quick Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-small-mizonal3-E5-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-small-mizonal3-E5-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-small-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-small-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-small-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-small-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-small-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.23	0.6567	0.5917	39.58	14.60	1.49e-04	7.39
500	0.46	0.7308	0.7316	45.99	26.32	2.99e-04	7.13
750	0.68	0.5756	0.6663	34.63	14.25	2.91e-04	4.82
1000	0.91	0.4654	0.6053	34.85	15.92	2.82e-04	4.24
1250	1.14	0.3519	0.6088	29.56	11.89	2.73e-04	4.16
1500	1.37	0.2995	0.6002	28.53	10.57	2.64e-04	3.88
1750	1.59	0.2729	0.5886	27.61	10.26	2.55e-04	2.98
2000	1.82	0.2157	0.5701	27.11	11.45	2.46e-04	3.24
2250	2.05	0.1739	0.5882	27.56	11.58	2.37e-04	2.42
2500	2.28	0.1598	0.6037	27.01	10.17	2.28e-04	2.60
2750	2.50	0.1459	0.5841	26.76	10.50	2.19e-04	1.83
3000	2.73	0.1313	0.6031	26.37	10.50	2.10e-04	1.71
3250	2.96	0.1114	0.6189	25.15	9.34	2.00e-04	1.42
3500	3.19	0.0947	0.6287	26.89	10.82	1.91e-04	1.46
3750	3.42	0.0854	0.6224	24.96	9.19	1.82e-04	1.21
4000	3.64	0.0801	0.6257	24.19	8.60	1.73e-04	1.37
4250	3.87	0.0754	0.6275	24.84	10.09	1.64e-04	1.38
4500	4.10	0.0536	0.6366	24.42	9.31	1.55e-04	1.81
4750	4.33	0.0480	0.6308	22.79	8.38	1.46e-04	1.78
5000	4.55	0.0445	0.6399	22.49	8.40	1.37e-04	0.92
5250	4.78	0.0418	0.6450	22.36	7.97	1.28e-04	1.82
5500	5.01	0.0335	0.6303	21.82	7.97	1.19e-04	0.80
5750	5.24	0.0264	0.6384	21.99	7.75	1.10e-04	0.57
6000	5.46	0.0256	0.6297	21.51	7.73	1.01e-04	0.95
6250	5.69	0.0215	0.6090	20.70	7.22	9.18e-05	0.55
6500	5.92	0.0206	0.6253	21.00	7.52	8.27e-05	0.69
6750	6.15	0.0129	0.6366	20.99	7.37	7.37e-05	0.88
7000	6.38	0.0139	0.6258	21.11	7.39	6.46e-05	0.66
7250	6.60	0.0123	0.6210	20.11	6.71	5.56e-05	0.35
7500	6.83	0.0095	0.6215	20.02	6.56	4.65e-05	0.43
7750	7.06	0.0070	0.6260	19.95	6.55	3.75e-05	0.48
8000	7.29	0.0067	0.6361	19.37	6.44	2.84e-05	0.27
8250	7.51	0.0054	0.6377	19.70	6.62	1.94e-05	0.05
8500	7.74	0.0056	0.6291	19.61	6.56	1.03e-05	0.59
8750	7.97	0.0050	0.6303	19.40	6.54	1.27e-06	0.27