Whisper Large V3 — Multilingual Luganda + Ugandan English NSS (ml-v1)
Stage 2 multilingual fine-tune on the CDLI Ugandan Nonstandard Speech datasets.
Built on top of lg_waxal_tune_whisper_large_7 (Stage 1).
Trained jointly on Luganda and Ugandan English NSS data. Evaluated on Luganda NSS test set.
Training Details
- Datasets: cdli/ugandan_luganda_nonstandard_speech_v1.0 + cdli/ugandan_english_nonstandard_speech_v1.0
- Stage 1 checkpoint: lg_waxal_tune_whisper_large_7/best_model
- Encoder frozen: No
- Decoder frozen: No
- Projection frozen: No
- SpecAugment: On
- Data augmentation: Yes (Gaussian noise, gain perturbation, speed perturbation)
- Learning rate: 5e-5 (polynomial decay)
- Max steps: 1000
- Best checkpoint: Step 850
Results on Luganda NSS Test Set (checkpoint-850)
| Severity |
Baseline WER |
Fine-tuned WER |
| Mild |
0.89 |
0.50 |
| Moderate |
0.96 |
0.53 |
| Severe |
0.97 |
0.60 |
| Etiology |
Baseline WER |
Fine-tuned WER |
| Acquired hearing impairment |
0.98 |
0.67 |
| Cerebral Palsy |
0.95 |
0.43 |
| Cerebral palsy - cerebral malaria |
0.93 |
0.55 |
| Developmental |
0.97 |
0.54 |
| Idiopathic |
0.80 |
0.45 |
| Structural (tongue tie) |
0.96 |
0.65 |
- Overall WER (normalized): 0.745
- Overall CER (normalized): 0.290
- Avg WER (normalized): 0.529
- Avg CER (normalized): 0.182
- Meaning preservation: 85.1%
