whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06

This model is a fine-tuned version of openai/whisper-large-v3-turbo on the MiZonal v3.0 dataset.

It achieves the following results on the evaluation set:

• Wer: 14.6167
• Cer: 2.8214
• Real Time Factor: 0.0560

Quick Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.4554	0.5879	0.4359	32.0230	7.0116	4.98e-06	11.2265
500	0.9107	0.3496	0.3265	22.7498	4.9071	9.98e-06	8.8566
750	1.3661	0.2324	0.2632	19.2858	3.9599	9.36e-06	7.9487
1000	1.8215	0.1762	0.2435	17.4954	3.6383	8.72e-06	5.7156
1250	2.2769	0.0953	0.2509	17.7873	4.6455	8.08e-06	4.9855
1500	2.7322	0.0857	0.2428	18.3711	5.0095	7.43e-06	4.0912
1750	3.1876	0.0474	0.2399	16.2304	3.3503	6.79e-06	2.3282
2000	3.6430	0.0421	0.2514	15.9774	4.0147	6.15e-06	2.2509
2250	4.0984	0.0224	0.2566	14.7903	3.1241	5.51e-06	1.3668
2500	4.5537	0.0202	0.2679	15.4520	3.3097	4.86e-06	2.0542
2750	5.0091	0.0164	0.2702	16.7072	5.0944	4.22e-06	1.1179
3000	5.4645	0.0107	0.2733	16.6780	5.0908	3.58e-06	2.6542
3250	5.9199	0.0076	0.2848	13.9729	3.0322	2.94e-06	0.2975
3500	6.3752	0.0039	0.2819	15.7731	5.0785	2.29e-06	0.1877
3750	6.8306	0.0024	0.2863	13.7297	2.8979	1.65e-06	0.3065
4000	7.2860	0.0015	0.2860	13.7297	2.8697	1.01e-06	0.2391
4250	7.7413	0.0010	0.2887	14.8779	4.2497	3.67e-07	0.0548