xlsr-300m-mizonal3-E5-lus-v2026.06

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

• Wer: 21.3311
• Cer: 4.1319
• Real Time Factor: 0.0025

Quick Inference

import torch
import librosa
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2Processor.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E5-lus-v2026.06")
model = Wav2Vec2ForCTC.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E5-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_values = processor(audio, sampling_rate=16000, return_tensors="pt").input_values.to(device)

with torch.no_grad():
    logits = model(input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/xlsr-300m-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/xlsr-300m-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/xlsr-300m-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/xlsr-300m-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/xlsr-300m-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.23	11.1973	2.5030	99.65	89.05	1.49e-04	4.93
500	0.46	3.6668	0.4349	46.45	11.74	2.99e-04	3.25
750	0.68	2.6013	0.2847	34.83	7.99	2.91e-04
1000	0.91	2.1367	0.2302	27.80	6.18	2.82e-04	2.98
1250	1.14	1.9568	0.2180	26.44	5.73	2.73e-04	3.49
1500	1.37	1.7271	0.1947	23.75	5.08	2.64e-04	2.91
1750	1.59	1.6789	0.1823	22.22	4.66	2.55e-04	3.28
2000	1.82	1.5410	0.1636	20.88	4.31	2.46e-04	2.74
2250	2.05	1.4111	0.1586	19.49	4.00	2.37e-04	1.92
2500	2.28	1.2955	0.1595	19.67	4.07	2.28e-04	2.52
2750	2.50	1.7424	0.2740	22.09	4.42	2.19e-04	1.73
3000	2.73	3.3783	0.6241	24.00	4.76	2.10e-04	2.23
3250	2.96	3.2508	0.5673	47.47	9.46	2.00e-04	1.61
3500	3.19	3.0280	0.4417	33.84	5.95	1.91e-04	2.10
3750	3.42	2.3210	0.3097	28.41	5.07	1.82e-04	0.61
4000	3.64	2.2703	0.2825	27.12	4.93	1.73e-04	2.67
4250	3.87	2.1037	0.2618	28.42	5.09	1.64e-04	3.04
4500	4.10	2.1371	0.2502	25.45	4.76	1.55e-04	4.24
4750	4.33	2.1388	0.2530	29.54	5.21	1.46e-04	0.68
5000	4.55	2.1726	0.2711	35.54	6.13	1.37e-04	0.32
5250	4.78	2.1541	0.2739	35.87	6.20	1.28e-04	0.00
5500	5.01	2.1398	0.2741	35.85	6.20	1.19e-04	0.00
5750	5.24	2.2197	0.2741	35.85	6.20	1.10e-04	0.00
6000	5.46	2.1714	0.2741	35.85	6.20	1.01e-04	0.00
6250	5.69	2.1648	0.2741	35.85	6.20	9.18e-05	0.00
6500	5.92	2.2376	0.2741	35.85	6.20	8.27e-05	0.00
6750	6.15	2.1807	0.2741	35.85	6.20	7.37e-05	0.00
7000	6.38	2.1587	0.2741	35.85	6.20	6.46e-05	0.00
7250	6.60	2.1262	0.2741	35.85	6.20	5.56e-05	0.00
7500	6.83	2.1163	0.2741	35.85	6.20	4.65e-05	0.00
7750	7.06	2.1850	0.2741	35.85	6.20	3.75e-05	0.00
8000	7.29	2.1801	0.2741	35.85	6.20	2.84e-05	0.00
8250	7.51	2.0928	0.2741	35.85	6.20	1.94e-05	0.00
8500	7.74	2.0858	0.2741	35.85	6.20	1.03e-05	0.00
8750	7.97	2.1298	0.2741	35.85	6.20	1.27e-06	0.00
8784	8.00		0.2741	35.85	6.20