whisper-small-mizonal3-E3-lus-v2026.06

This model is a fine-tuned version of openai/whisper-small on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

• Wer: 20.4967
• Cer: 6.4709
• Real Time Factor: 0.0243

Quick Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-small-mizonal3-E3-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-small-mizonal3-E3-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-small-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-small-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-small-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-small-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-small-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.30	0.5117	0.6048	42.92	23.23	1.49e-04	8.18
500	0.61	0.5902	0.7649	41.97	22.09	2.99e-04	6.76
750	0.91	0.4742	0.6762	36.70	14.19	2.88e-04	5.18
1000	1.21	0.3240	0.6438	34.36	14.20	2.75e-04	4.26
1250	1.52	0.2754	0.6361	30.61	11.94	2.63e-04	5.36
1500	1.82	0.2209	0.6062	29.37	11.53	2.51e-04	3.36
1750	2.12	0.1560	0.6066	32.15	14.33	2.38e-04	2.49
2000	2.43	0.1419	0.6140	28.50	11.63	2.26e-04	3.06
2250	2.73	0.1232	0.5859	26.34	9.20	2.14e-04	1.86
2500	3.03	0.0846	0.6161	25.75	9.01	2.02e-04	1.35
2750	3.34	0.0802	0.6025	25.76	8.96	1.89e-04	1.34
3000	3.64	0.0765	0.6142	24.79	8.94	1.77e-04	1.46
3250	3.94	0.0528	0.6135	23.69	8.25	1.65e-04	1.27
3500	4.25	0.0430	0.6226	23.66	7.70	1.52e-04	1.20
3750	4.55	0.0378	0.6154	23.20	8.20	1.40e-04	0.98
4000	4.85	0.0313	0.5953	22.71	8.03	1.28e-04	0.85
4250	5.16	0.0218	0.5940	21.39	7.27	1.15e-04	1.36
4500	5.46	0.0122	0.5950	21.27	7.09	1.03e-04	0.51
4750	5.77	0.0107	0.5929	20.52	6.82	9.08e-05	0.73
5000	6.07	0.0049	0.5837	19.62	6.21	7.84e-05	0.14
5250	6.37	0.0036	0.5987	19.14	5.97	6.61e-05	0.55
5500	6.68	0.0035	0.5952	19.12	6.03	5.38e-05	0.29
5750	6.98	0.0015	0.6068	18.79	5.80	4.15e-05	0.09
6000	7.28	0.0013	0.6160	18.92	5.82	2.92e-05	0.00
6250	7.59	0.0004	0.6217	18.94	5.78	1.69e-05	0.02
6500	7.89	0.0006	0.6260	18.90	5.73	4.58e-06	0.00