--- language: - lus license: apache-2.0 pipeline_tag: automatic-speech-recognition base_model: openai/whisper-medium tags: - generated_from_trainer datasets: - andrewbawitlung/MiZonal-v3.0 metrics: - wer - cer model-index: - name: whisper-medium-mizonal3-E5-lus-v2026.06 results: - task: name: Automatic Speech Recognition type: automatic-speech-recognition dataset: name: MiZonal v3.0 type: andrewbawitlung/MiZonal-v3.0 config: default split: test metrics: - name: Wer type: wer value: 24.5705 - name: Cer type: cer value: 9.3880 - name: Real Time Factor type: rtf value: 0.0531 --- ![Mizo Automatic Speech Recognition (ASR) Models v3.0](banner.jpg) # whisper-medium-mizonal3-E5-lus-v2026.06 This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on the **MiZonal v3.0** dataset. Note: ~1 hour of conversational speech was added to this dataset version. It achieves the following results on the evaluation set: - Wer: 24.5705 - Cer: 9.3880 - Real Time Factor: 0.0531 ## Quick Inference ```python import torch import librosa from transformers import WhisperProcessor, WhisperForConditionalGeneration device = "cuda" if torch.cuda.is_available() else "cpu" processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06") model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06").to(device) audio, sr = librosa.load("your_audio.wav", sr=16000) input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device) with torch.no_grad(): predicted_ids = model.generate(input_features, max_new_tokens=256) transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0] print(transcription) ``` ## Model description ### Experiment Configurations This repository is part of a series of experiments. The different configurations are: - **E1 (Baseline):** Standard training configuration. - **E2 (Noise):** Training with background noise augmentation. - **E3 (Speed):** Training with speed perturbation augmentation. - **E4 (SpecAug):** Training with SpecAugment (time and frequency masking). - **E5 (Combined):** Training with a combination of all augmentations. ### All Models in this Family | Experiment | Hugging Face Repository | | :--- | :--- | | **E1 (Baseline)** | [andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06) | | **E2 (Noise)** | [andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06) | | **E3 (Speed)** | [andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06) | | **E4 (SpecAug)** | [andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06) | | **E5 (Combined)** | [andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06) | ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0003 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: OptimizerNames.ADAMW_TORCH_FUSED - lr_scheduler_type: SchedulerType.LINEAR - num_epochs: 8 ### Training results | step | epoch | train_loss | eval_loss | eval_wer | eval_cer | learning_rate | grad_norm | | --- | --- | --- | --- | --- | --- | --- | --- | | 250 | 0.23 | 0.6933 | 0.6139 | 40.78 | 16.18 | 1.49e-04 | 6.36 | | 500 | 0.46 | 0.9767 | 0.8149 | 47.59 | 26.86 | 2.99e-04 | 6.31 | | 750 | 0.68 | 0.7091 | 0.7418 | 41.24 | 17.77 | 2.91e-04 | 5.07 | | 1000 | 0.91 | 0.6460 | 0.7217 | 37.89 | 15.42 | 2.82e-04 | 5.18 | | 1250 | 1.14 | 0.4395 | 0.6853 | 36.05 | 16.92 | 2.73e-04 | 4.28 | | 1500 | 1.37 | 0.3875 | 0.7011 | 37.08 | 18.51 | 2.64e-04 | 2.90 | | 1750 | 1.59 | 0.3301 | 0.6582 | 35.70 | 17.54 | 2.55e-04 | 3.33 | | 2000 | 1.82 | 0.2798 | 0.6510 | 33.88 | 15.94 | 2.46e-04 | 2.80 | | 2250 | 2.05 | 0.2123 | 0.6416 | 35.43 | 18.46 | 2.37e-04 | 2.66 | | 2500 | 2.28 | 0.1885 | 0.6682 | 49.61 | 34.99 | 2.28e-04 | 2.33 | | 2750 | 2.50 | 0.1734 | 0.6513 | 31.96 | 15.30 | 2.19e-04 | 2.27 | | 3000 | 2.73 | 0.1532 | 0.6563 | 31.40 | 15.44 | 2.10e-04 | 2.93 | | 3250 | 2.96 | 0.1287 | 0.6604 | 48.16 | 34.82 | 2.00e-04 | 1.70 | | 3500 | 3.19 | 0.1061 | 0.6573 | 44.59 | 30.61 | 1.91e-04 | 1.50 | | 3750 | 3.42 | 0.0947 | 0.6516 | 34.12 | 17.72 | 1.82e-04 | 1.65 | | 4000 | 3.64 | 0.0940 | 0.6546 | 29.34 | 13.25 | 1.73e-04 | 1.65 | | 4250 | 3.87 | 0.0791 | 0.6389 | 32.56 | 16.25 | 1.64e-04 | 1.77 | | 4500 | 4.10 | 0.0649 | 0.6551 | 37.08 | 21.99 | 1.55e-04 | 1.24 | | 4750 | 4.33 | 0.0573 | 0.6678 | 33.99 | 19.43 | 1.46e-04 | 0.84 | | 5000 | 4.55 | 0.0546 | 0.6536 | 27.46 | 12.05 | 1.37e-04 | 1.02 | | 5250 | 4.78 | 0.0469 | 0.6603 | 28.65 | 13.21 | 1.28e-04 | 1.11 | | 5500 | 5.01 | 0.0390 | 0.6679 | 29.75 | 15.28 | 1.19e-04 | 0.97 | | 5750 | 5.24 | 0.0315 | 0.6669 | 28.00 | 13.77 | 1.10e-04 | 0.53 | | 6000 | 5.46 | 0.0270 | 0.6814 | 32.93 | 17.82 | 1.01e-04 | 0.68 | | 6250 | 5.69 | 0.0236 | 0.6527 | 29.96 | 16.07 | 9.18e-05 | 0.50 | | 6500 | 5.92 | 0.0220 | 0.6690 | 23.19 | 9.03 | 8.27e-05 | 0.70 | | 6750 | 6.15 | 0.0166 | 0.6889 | 25.64 | 11.19 | 7.37e-05 | 0.90 | | 7000 | 6.38 | 0.0125 | 0.6636 | 27.15 | 12.85 | 6.46e-05 | 0.51 | | 7250 | 6.60 | 0.0097 | 0.6960 | 29.56 | 15.38 | 5.56e-05 | 0.44 | | 7500 | 6.83 | 0.0101 | 0.6943 | 30.11 | 16.66 | 4.65e-05 | 0.65 | | 7750 | 7.06 | 0.0076 | 0.6850 | 27.00 | 13.23 | 3.75e-05 | 0.82 | | 8000 | 7.29 | 0.0072 | 0.6907 | 25.29 | 11.81 | 2.84e-05 | 0.10 | | 8250 | 7.51 | 0.0048 | 0.6933 | 27.07 | 13.31 | 1.94e-05 | 0.05 | | 8500 | 7.74 | 0.0044 | 0.6936 | 25.82 | 12.25 | 1.03e-05 | 0.26 | | 8750 | 7.97 | 0.0046 | 0.6891 | 25.88 | 12.37 | 1.27e-06 | 0.24 |