whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06

This model is a fine-tuned version of openai/whisper-large-v3-turbo on the MiZonal v3.0 dataset.

It achieves the following results on the evaluation set:

• Wer: 14.1847
• Cer: 2.7327
• Real Time Factor: 0.0546

Quick Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.3036	0.4524	0.4416	30.3882	7.1247	4.98e-06	10.7001
500	0.6072	0.2872	0.3239	21.8449	4.8752	9.98e-06	8.0670
750	0.9107	0.1944	0.2786	18.7895	3.9847	9.59e-06	4.7649
1000	1.2137	0.1211	0.2689	20.6967	6.1493	9.18e-06	4.6896
1250	1.5173	0.1010	0.2581	18.7311	6.0362	8.77e-06	4.2444
1500	1.8209	0.0740	0.2458	20.3269	8.8705	8.36e-06	3.2823
1750	2.1239	0.0473	0.2598	17.3883	3.5464	7.95e-06	3.7458
2000	2.4274	0.0402	0.2674	16.4445	4.1384	7.54e-06	2.7577
2250	2.7310	0.0334	0.2735	15.8801	3.3309	7.13e-06	2.9316
2500	3.0340	0.0208	0.2770	15.0141	3.1807	6.72e-06	1.7462
2750	3.3376	0.0188	0.2794	15.0238	3.2072	6.31e-06	1.9191
3000	3.6412	0.0133	0.2769	15.8996	3.7249	5.90e-06	1.2823
3250	3.9447	0.0139	0.2932	15.1309	3.3821	5.49e-06	2.2204
3500	4.2477	0.0084	0.2984	15.1114	3.2213	5.08e-06	0.8746
3750	4.5513	0.0080	0.2949	14.9849	3.4386	4.67e-06	1.8859
4000	4.8549	0.0069	0.2994	14.4595	3.0711	4.26e-06	1.2093
4250	5.1579	0.0034	0.2979	14.3622	2.9951	3.85e-06	0.9255
4500	5.4614	0.0032	0.2984	14.6638	3.6295	3.44e-06	0.0719
4750	5.7650	0.0024	0.3041	13.9243	2.8626	3.03e-06	0.0865
5000	6.0680	0.0016	0.3079	13.9535	2.8573	2.61e-06	0.0794
5250	6.3716	0.0016	0.3037	13.7102	2.8343	2.20e-06	0.1663
5500	6.6752	0.0006	0.3071	13.9632	3.4174	1.79e-06	0.5288
5750	6.9787	0.0011	0.3134	14.3524	3.5252	1.38e-06	0.0395
6000	7.2817	0.0012	0.3131	14.0897	3.5146	9.73e-07	0.0409
6250	7.5853	0.0004	0.3130	13.4572	2.7672	5.63e-07	0.0160
6500	7.8889	0.0006	0.3144	13.4670	2.7707	1.53e-07	0.0153