whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06

This model is a fine-tuned version of openai/whisper-large-v3-turbo on the MiZonal v3.0 dataset.

It achieves the following results on the evaluation set:

Wer: 15.2744
Cer: 2.9739
Real Time Factor: 0.0546

Quick Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

E1 (Baseline): Standard training configuration.
E2 (Noise): Training with background noise augmentation.
E3 (Speed): Training with speed perturbation augmentation.
E4 (SpecAug): Training with SpecAugment (time and frequency masking).
E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: OptimizerNames.ADAMW_TORCH_FUSED
lr_scheduler_type: SchedulerType.LINEAR
num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.9107	0.5478	0.4402	30.4953	7.0045	4.98e-06	9.9833
500	1.8197	0.3494	0.3157	22.3995	4.7993	9.98e-06	8.1049
750	2.7286	0.2393	0.2534	19.2079	3.9317	8.54e-06	6.5961
1000	3.6375	0.1636	0.2337	16.6196	3.4139	7.06e-06	6.5331
1250	4.5464	0.1076	0.2236	15.5687	3.1418	5.59e-06	4.9694
1500	5.4554	0.0743	0.2219	17.9722	5.5450	4.12e-06	4.4744
1750	6.3643	0.0506	0.2183	14.6735	2.9262	2.65e-06	3.7141
2000	7.2732	0.0339	0.2199	13.8465	2.7654	1.18e-06	2.8581