---
language:
- lus
license: apache-2.0
pipeline_tag: automatic-speech-recognition
base_model: openai/whisper-medium
tags:
- generated_from_trainer
datasets:
- andrewbawitlung/MiZonal-v3.0
metrics:
- wer
- cer
model-index:
- name: whisper-medium-mizonal3-E3-lus-v2026.06
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: MiZonal v3.0
      type: andrewbawitlung/MiZonal-v3.0
      config: default
      split: test
    metrics:
    - name: Wer
      type: wer
      value: 26.3473
    - name: Cer
      type: cer
      value: 10.6790
    - name: Real Time Factor
      type: rtf
      value: 0.0546
---
![Mizo Automatic Speech Recognition (ASR) Models v3.0](banner.jpg)

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# whisper-medium-mizonal3-E3-lus-v2026.06

This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on the **MiZonal v3.0** dataset. 
Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:
- Wer: 26.3473
- Cer: 10.6790
- Real Time Factor: 0.0546

## Quick Inference


```python
import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_new_tokens=256)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)
```


## Model description

### Experiment Configurations
This repository is part of a series of experiments. The different configurations are:
- **E1 (Baseline):** Standard training configuration.
- **E2 (Noise):** Training with background noise augmentation.
- **E3 (Speed):** Training with speed perturbation augmentation.
- **E4 (SpecAug):** Training with SpecAugment (time and frequency masking).
- **E5 (Combined):** Training with a combination of all augmentations.

### All Models in this Family
| Experiment | Hugging Face Repository |
| :--- | :--- |
| **E1 (Baseline)** | [andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06) |
| **E2 (Noise)** | [andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06) |
| **E3 (Speed)** | [andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06) |
| **E4 (SpecAug)** | [andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06) |
| **E5 (Combined)** | [andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06) |

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: OptimizerNames.ADAMW_TORCH_FUSED
- lr_scheduler_type: SchedulerType.LINEAR
- num_epochs: 8

### Training results

| step | epoch | train_loss | eval_loss | eval_wer | eval_cer | learning_rate | grad_norm |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 250 | 0.30 | 0.5439 | 0.6491 | 46.75 | 21.78 | 1.49e-04 | 6.48 |
| 500 | 0.61 | 2.0455 | 1.4417 | 64.19 | 40.93 | 2.99e-04 | 9.23 |
| 750 | 0.91 | 0.6062 | 0.8026 | 79.31 | 68.75 | 2.88e-04 | 5.40 |
| 1000 | 1.21 | 0.4792 | 0.7684 | 49.83 | 27.45 | 2.75e-04 | 4.66 |
| 1250 | 1.52 | 0.3800 | 0.6988 | 43.12 | 23.67 | 2.63e-04 | 4.61 |
| 1500 | 1.82 | 0.2845 | 0.6733 | 32.34 | 13.24 | 2.51e-04 | 3.43 |
| 1750 | 2.12 | 0.2145 | 0.7059 | 37.75 | 17.47 | 2.38e-04 | 2.72 |
| 2000 | 2.43 | 0.1927 | 0.6659 | 50.14 | 36.39 | 2.26e-04 | 2.25 |
| 2250 | 2.73 | 0.1722 | 0.6448 | 36.63 | 18.20 | 2.14e-04 | 2.69 |
| 2500 | 3.03 | 0.1153 | 0.6828 | 44.36 | 29.13 | 2.02e-04 | 1.91 |
| 2750 | 3.34 | 0.1133 | 0.6471 | 33.38 | 15.92 | 1.89e-04 | 1.97 |
| 3000 | 3.64 | 0.0923 | 0.6713 | 43.23 | 28.74 | 1.77e-04 | 2.38 |
| 3250 | 3.94 | 0.0774 | 0.6387 | 30.29 | 13.36 | 1.65e-04 | 1.54 |
| 3500 | 4.25 | 0.0568 | 0.6538 | 34.51 | 18.21 | 1.52e-04 | 1.37 |
| 3750 | 4.55 | 0.0498 | 0.6446 | 27.02 | 11.38 | 1.40e-04 | 1.09 |
| 4000 | 4.85 | 0.0387 | 0.6233 | 38.90 | 26.54 | 1.28e-04 | 1.08 |
| 4250 | 5.16 | 0.0242 | 0.6635 | 27.20 | 12.89 | 1.15e-04 | 0.97 |
| 4500 | 5.46 | 0.0216 | 0.6543 | 37.99 | 25.78 | 1.03e-04 | 0.60 |
| 4750 | 5.77 | 0.0136 | 0.6579 | 24.79 | 10.17 | 9.08e-05 | 0.60 |
| 5000 | 6.07 | 0.0041 | 0.6616 | 25.60 | 11.76 | 7.84e-05 | 0.06 |
| 5250 | 6.37 | 0.0044 | 0.6697 | 28.33 | 14.31 | 6.61e-05 | 0.01 |
| 5500 | 6.68 | 0.0047 | 0.6433 | 31.88 | 18.58 | 5.38e-05 | 0.31 |
| 5750 | 6.98 | 0.0030 | 0.6796 | 27.01 | 13.44 | 4.15e-05 | 0.02 |
| 6000 | 7.28 | 0.0016 | 0.6911 | 27.43 | 14.06 | 2.92e-05 | 0.04 |
| 6250 | 7.59 | 0.0007 | 0.7047 | 26.63 | 13.25 | 1.69e-05 | 0.01 |
| 6500 | 7.89 | 0.0007 | 0.7151 | 26.25 | 12.78 | 4.58e-06 | 0.00 |