---
language:
- lus
license: apache-2.0
pipeline_tag: automatic-speech-recognition
base_model: openai/whisper-large-v3-turbo
tags:
- generated_from_trainer
datasets:
- andrewbawitlung/MiZonal-v3.0
metrics:
- wer
- cer
model-index:
- name: whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: MiZonal v3.0
      type: andrewbawitlung/MiZonal-v3.0
      config: default
      split: test
    metrics:
    - name: Wer
      type: wer
      value: 13.8215
    - name: Cer
      type: cer
      value: 2.6263
    - name: Real Time Factor
      type: rtf
      value: 0.0540
---
![Mizo Automatic Speech Recognition (ASR) Models v3.0](banner.jpg)

# whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06

This model is a fine-tuned version of [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo) on the **MiZonal v3.0** dataset.

It achieves the following results on the evaluation set:
- Wer: 13.8215
- Cer: 2.6263
- Real Time Factor: 0.0540

## Quick Inference


```python
import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)
```


## Model description

### Experiment Configurations
This repository is part of a series of experiments. The different configurations are:
- **E1 (Baseline):** Standard training configuration.
- **E2 (Noise):** Training with background noise augmentation.
- **E3 (Speed):** Training with speed perturbation augmentation.
- **E4 (SpecAug):** Training with SpecAugment (time and frequency masking).
- **E5 (Combined):** Training with a combination of all augmentations.

### All Models in this Family
| Experiment | Hugging Face Repository |
| :--- | :--- |
| **E1 (Baseline)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06) |
| **E2 (Noise)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06) |
| **E3 (Speed)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06) |
| **E4 (SpecAug)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06) |
| **E5 (Combined)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06) |

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: OptimizerNames.ADAMW_TORCH_FUSED
- lr_scheduler_type: SchedulerType.LINEAR
- num_epochs: 8

### Training results

| step | epoch | train_loss | eval_loss | eval_wer | eval_cer | learning_rate | grad_norm |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 250 | 0.2277 | 0.6324 | 0.4640 | 31.7797 | 7.3367 | 4.98e-06 | 12.9801 |
| 500 | 0.4554 | 0.3951 | 0.3165 | 23.9078 | 5.1262 | 9.98e-06 | 9.2164 |
| 750 | 0.6831 | 0.2909 | 0.2734 | 19.9085 | 4.1225 | 9.70e-06 | 9.4305 |
| 1000 | 0.9107 | 0.2216 | 0.2501 | 18.0695 | 3.7514 | 9.40e-06 | 7.3999 |
| 1250 | 1.1384 | 0.1571 | 0.2263 | 16.0650 | 3.2867 | 9.10e-06 | 5.7364 |
| 1500 | 1.3661 | 0.1384 | 0.2388 | 15.9288 | 3.2973 | 8.79e-06 | 7.9211 |
| 1750 | 1.5938 | 0.1267 | 0.2285 | 15.3158 | 3.2460 | 8.49e-06 | 6.6193 |
| 2000 | 1.8215 | 0.1026 | 0.2304 | 15.2379 | 3.1277 | 8.19e-06 | 4.8834 |
| 2250 | 2.0492 | 0.0686 | 0.2323 | 15.8217 | 3.9705 | 7.89e-06 | 2.6450 |
| 2500 | 2.2769 | 0.0681 | 0.2267 | 14.9363 | 3.1612 | 7.59e-06 | 5.5804 |
| 2750 | 2.5046 | 0.0632 | 0.2334 | 14.6346 | 2.9828 | 7.29e-06 | 4.5015 |
| 3000 | 2.7322 | 0.0592 | 0.2377 | 13.9340 | 2.7831 | 6.98e-06 | 3.9216 |
| 3250 | 2.9599 | 0.0523 | 0.2398 | 13.9535 | 2.9015 | 6.68e-06 | 1.9478 |
| 3500 | 3.1876 | 0.0386 | 0.2425 | 14.7417 | 3.0446 | 6.38e-06 | 2.8374 |
| 3750 | 3.4153 | 0.0366 | 0.2519 | 14.4108 | 2.8520 | 6.08e-06 | 3.2853 |
| 4000 | 3.6430 | 0.0324 | 0.2560 | 14.4789 | 2.9704 | 5.78e-06 | 3.7112 |
| 4250 | 3.8707 | 0.0334 | 0.2457 | 13.3599 | 2.7053 | 5.47e-06 | 4.6839 |
| 4500 | 4.0984 | 0.0270 | 0.2491 | 13.7394 | 2.8750 | 5.17e-06 | 1.2767 |
| 4750 | 4.3260 | 0.0250 | 0.2480 | 13.2432 | 2.7106 | 4.87e-06 | 2.7142 |
| 5000 | 4.5537 | 0.0249 | 0.2555 | 13.5351 | 2.7477 | 4.57e-06 | 1.4408 |
| 5250 | 4.7814 | 0.0219 | 0.2582 | 14.0411 | 3.0481 | 4.27e-06 | 2.2743 |
| 5500 | 5.0091 | 0.0230 | 0.2572 | 13.0388 | 2.6894 | 3.97e-06 | 4.3055 |
| 5750 | 5.2368 | 0.0191 | 0.2595 | 13.8951 | 2.9032 | 3.66e-06 | 1.9485 |
| 6000 | 5.4645 | 0.0172 | 0.2617 | 13.0096 | 2.6771 | 3.36e-06 | 2.3269 |
| 6250 | 5.6922 | 0.0167 | 0.2556 | 13.2140 | 2.7901 | 3.06e-06 | 1.3072 |
| 6500 | 5.9199 | 0.0159 | 0.2591 | 12.6593 | 2.7018 | 2.76e-06 | 1.2552 |
| 6750 | 6.1475 | 0.0144 | 0.2653 | 13.0972 | 2.6276 | 2.46e-06 | 1.5865 |
| 7000 | 6.3752 | 0.0127 | 0.2578 | 12.7664 | 2.5587 | 2.15e-06 | 1.7085 |
| 7250 | 6.6029 | 0.0098 | 0.2672 | 12.8442 | 2.5887 | 1.85e-06 | 0.7853 |
| 7500 | 6.8306 | 0.0116 | 0.2598 | 12.5523 | 2.4880 | 1.55e-06 | 0.7284 |
| 7750 | 7.0583 | 0.0105 | 0.2603 | 12.6885 | 2.5463 | 1.25e-06 | 1.8309 |
| 8000 | 7.2860 | 0.0085 | 0.2628 | 12.5523 | 2.4809 | 9.48e-07 | 2.5387 |
| 8250 | 7.5137 | 0.0087 | 0.2628 | 12.6885 | 2.5357 | 6.46e-07 | 1.0704 |
| 8500 | 7.7413 | 0.0057 | 0.2610 | 12.5815 | 2.5198 | 3.44e-07 | 0.9892 |
| 8750 | 7.9690 | 0.0091 | 0.2628 | 12.5620 | 2.5269 | 4.23e-08 | 0.6342 |