xlsr-300m-mizonal3-E3-lus-v2026.06

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

• Wer: 20.4476
• Cer: 4.0148
• Real Time Factor: 0.0025

Quick Inference

import torch
import librosa
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2Processor.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E3-lus-v2026.06")
model = Wav2Vec2ForCTC.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E3-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_values = processor(audio, sampling_rate=16000, return_tensors="pt").input_values.to(device)

with torch.no_grad():
    logits = model(input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/xlsr-300m-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/xlsr-300m-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/xlsr-300m-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/xlsr-300m-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/xlsr-300m-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.30	10.6431	2.1604	100.00	72.37	1.49e-04	5.71
500	0.61	2.1782	0.4132	44.76	11.13	2.99e-04	2.48
750	0.91	1.3981	0.2759	31.78	7.26	2.88e-04	2.89
1000	1.21	0.9750	0.2297	27.19	5.93	2.75e-04	2.30
1250	1.52	0.8140	0.2025	24.25	5.02	2.63e-04	1.35
1500	1.82	0.6315	0.1989	22.43	4.70	2.51e-04	1.10
1750	2.12	0.5285	0.1901	21.77	4.46	2.38e-04	1.24
2000	2.43	0.4376	0.1880	19.81	4.16	2.26e-04	1.38
2250	2.73	0.4403	0.1728	19.28	3.89	2.14e-04	1.99
2500	3.03	0.3461	0.1761	18.50	3.81	2.02e-04	0.74
2750	3.34	0.3648	0.1698	19.05	3.98	1.89e-04	0.88
3000	3.64	0.4385	0.2316	19.31	3.94	1.77e-04	1.42
3250	3.94	1.5365	0.4190	25.32	5.43	1.65e-04	1.50
3500	4.25	2.1459	0.4656	24.78	5.30	1.52e-04	1.19
3750	4.55	1.1671	0.2744	19.70	3.96	1.40e-04	0.92
4000	4.85	0.7579	0.2133	19.52	3.90	1.28e-04	0.96
4250	5.16	0.6674	0.2029	19.01	3.86	1.15e-04	1.30
4500	5.46	0.6051	0.2036	18.82	3.82	1.03e-04	0.96
4750	5.77	0.6231	0.1953	18.59	3.79	9.08e-05	0.36
5000	6.07	0.6137	0.1941	18.85	3.82	7.84e-05	0.31
5250	6.37	0.6259	0.1948	18.78	3.79	6.61e-05	0.12
5500	6.68	0.6252	0.1939	19.18	3.87	5.38e-05	0.00
5750	6.98	0.6704	0.1939	19.20	3.87	4.15e-05	0.00
6000	7.28	0.6658	0.1939	19.20	3.87	2.92e-05	0.00
6250	7.59	0.6350	0.1939	19.20	3.87	1.69e-05	0.00
6500	7.89	0.6572	0.1939	19.20	3.87	4.58e-06	0.00
6592	8.00		0.1939	19.20	3.87