File size: 4,356 Bytes
7341b40 75ed02c 7341b40 5a8cb2f 7341b40 5a8cb2f 7341b40 6abd496 7341b40 5a8cb2f 7341b40 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | ---
language:
- lus
license: apache-2.0
pipeline_tag: automatic-speech-recognition
base_model: openai/whisper-large-v3-turbo
tags:
- generated_from_trainer
datasets:
- andrewbawitlung/MiZonal-v3.0
metrics:
- wer
- cer
model-index:
- name: whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06
results:
- task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: MiZonal v3.0
type: andrewbawitlung/MiZonal-v3.0
config: default
split: test
metrics:
- name: Wer
type: wer
value: 15.2744
- name: Cer
type: cer
value: 2.9739
- name: Real Time Factor
type: rtf
value: 0.0546
---
# whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06
This model is a fine-tuned version of [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo) on the **MiZonal v3.0** dataset.
It achieves the following results on the evaluation set:
- Wer: 15.2744
- Cer: 2.9739
- Real Time Factor: 0.0546
## Quick Inference
```python
import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06").to(device)
audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)
with torch.no_grad():
predicted_ids = model.generate(input_features, max_new_tokens=256)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)
```
## Model description
### Experiment Configurations
This repository is part of a series of experiments. The different configurations are:
- **E1 (Baseline):** Standard training configuration.
- **E2 (Noise):** Training with background noise augmentation.
- **E3 (Speed):** Training with speed perturbation augmentation.
- **E4 (SpecAug):** Training with SpecAugment (time and frequency masking).
- **E5 (Combined):** Training with a combination of all augmentations.
### All Models in this Family
| Experiment | Hugging Face Repository |
| :--- | :--- |
| **E1 (Baseline)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E1-lus-v2026.06) |
| **E2 (Noise)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E2-lus-v2026.06) |
| **E3 (Speed)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E3-lus-v2026.06) |
| **E4 (SpecAug)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E4-lus-v2026.06) |
| **E5 (Combined)** | [andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06](https://huggingface.co/andrewbawitlung/whisper-large-v3-turbo-mizonal3-E5-lus-v2026.06) |
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: OptimizerNames.ADAMW_TORCH_FUSED
- lr_scheduler_type: SchedulerType.LINEAR
- num_epochs: 8
### Training results
| step | epoch | train_loss | eval_loss | eval_wer | eval_cer | learning_rate | grad_norm |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 250 | 0.9107 | 0.5478 | 0.4402 | 30.4953 | 7.0045 | 4.98e-06 | 9.9833 |
| 500 | 1.8197 | 0.3494 | 0.3157 | 22.3995 | 4.7993 | 9.98e-06 | 8.1049 |
| 750 | 2.7286 | 0.2393 | 0.2534 | 19.2079 | 3.9317 | 8.54e-06 | 6.5961 |
| 1000 | 3.6375 | 0.1636 | 0.2337 | 16.6196 | 3.4139 | 7.06e-06 | 6.5331 |
| 1250 | 4.5464 | 0.1076 | 0.2236 | 15.5687 | 3.1418 | 5.59e-06 | 4.9694 |
| 1500 | 5.4554 | 0.0743 | 0.2219 | 17.9722 | 5.5450 | 4.12e-06 | 4.4744 |
| 1750 | 6.3643 | 0.0506 | 0.2183 | 14.6735 | 2.9262 | 2.65e-06 | 3.7141 |
| 2000 | 7.2732 | 0.0339 | 0.2199 | 13.8465 | 2.7654 | 1.18e-06 | 2.8581 |
|