whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06

This model is a fine-tuned version of openai/whisper-large-v3-turbo on the MiZonal v3.0 dataset.

It achieves the following results on the evaluation set:

• Wer: 13.8215
• Cer: 2.6263
• Real Time Factor: 0.0540

Quick Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.2277	0.6324	0.4640	31.7797	7.3367	4.98e-06	12.9801
500	0.4554	0.3951	0.3165	23.9078	5.1262	9.98e-06	9.2164
750	0.6831	0.2909	0.2734	19.9085	4.1225	9.70e-06	9.4305
1000	0.9107	0.2216	0.2501	18.0695	3.7514	9.40e-06	7.3999
1250	1.1384	0.1571	0.2263	16.0650	3.2867	9.10e-06	5.7364
1500	1.3661	0.1384	0.2388	15.9288	3.2973	8.79e-06	7.9211
1750	1.5938	0.1267	0.2285	15.3158	3.2460	8.49e-06	6.6193
2000	1.8215	0.1026	0.2304	15.2379	3.1277	8.19e-06	4.8834
2250	2.0492	0.0686	0.2323	15.8217	3.9705	7.89e-06	2.6450
2500	2.2769	0.0681	0.2267	14.9363	3.1612	7.59e-06	5.5804
2750	2.5046	0.0632	0.2334	14.6346	2.9828	7.29e-06	4.5015
3000	2.7322	0.0592	0.2377	13.9340	2.7831	6.98e-06	3.9216
3250	2.9599	0.0523	0.2398	13.9535	2.9015	6.68e-06	1.9478
3500	3.1876	0.0386	0.2425	14.7417	3.0446	6.38e-06	2.8374
3750	3.4153	0.0366	0.2519	14.4108	2.8520	6.08e-06	3.2853
4000	3.6430	0.0324	0.2560	14.4789	2.9704	5.78e-06	3.7112
4250	3.8707	0.0334	0.2457	13.3599	2.7053	5.47e-06	4.6839
4500	4.0984	0.0270	0.2491	13.7394	2.8750	5.17e-06	1.2767
4750	4.3260	0.0250	0.2480	13.2432	2.7106	4.87e-06	2.7142
5000	4.5537	0.0249	0.2555	13.5351	2.7477	4.57e-06	1.4408
5250	4.7814	0.0219	0.2582	14.0411	3.0481	4.27e-06	2.2743
5500	5.0091	0.0230	0.2572	13.0388	2.6894	3.97e-06	4.3055
5750	5.2368	0.0191	0.2595	13.8951	2.9032	3.66e-06	1.9485
6000	5.4645	0.0172	0.2617	13.0096	2.6771	3.36e-06	2.3269
6250	5.6922	0.0167	0.2556	13.2140	2.7901	3.06e-06	1.3072
6500	5.9199	0.0159	0.2591	12.6593	2.7018	2.76e-06	1.2552
6750	6.1475	0.0144	0.2653	13.0972	2.6276	2.46e-06	1.5865
7000	6.3752	0.0127	0.2578	12.7664	2.5587	2.15e-06	1.7085
7250	6.6029	0.0098	0.2672	12.8442	2.5887	1.85e-06	0.7853
7500	6.8306	0.0116	0.2598	12.5523	2.4880	1.55e-06	0.7284
7750	7.0583	0.0105	0.2603	12.6885	2.5463	1.25e-06	1.8309
8000	7.2860	0.0085	0.2628	12.5523	2.4809	9.48e-07	2.5387
8250	7.5137	0.0087	0.2628	12.6885	2.5357	6.46e-07	1.0704
8500	7.7413	0.0057	0.2610	12.5815	2.5198	3.44e-07	0.9892
8750	7.9690	0.0091	0.2628	12.5620	2.5269	4.23e-08	0.6342