xlsr-300m-mizonal3-E2-lus-v2026.06

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

• Wer: 20.3887
• Cer: 4.0148
• Real Time Factor: 0.0024

Quick Inference

import torch
import librosa
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2Processor.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E2-lus-v2026.06")
model = Wav2Vec2ForCTC.from_pretrained("andrewbawitlung/xlsr-300m-mizonal3-E2-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_values = processor(audio, sampling_rate=16000, return_tensors="pt").input_values.to(device)

with torch.no_grad():
    logits = model(input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/xlsr-300m-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/xlsr-300m-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/xlsr-300m-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/xlsr-300m-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/xlsr-300m-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.46	11.1325	2.5736	100.00	100.00	1.49e-04	1.56
500	0.91	2.6765	0.4443	47.05	11.73	2.99e-04	2.91
750	1.37	1.7642	0.2865	34.07	7.75	2.81e-04	2.16
1000	1.82	1.2532	0.2242	27.54	5.90	2.62e-04	2.45
1250	2.28	0.9380	0.2079	25.66	5.39	2.42e-04	1.79
1500	2.73	0.7964	0.1916	23.73	4.82	2.23e-04	1.96
1750	3.19	0.6060	0.1748	20.62	4.29	2.04e-04	1.73
2000	3.64	0.6421	0.1647	19.33	3.94	1.84e-04	1.21
2250	4.10	0.4956	0.1723	19.03	3.88	1.65e-04	1.17
2500	4.55	0.6000	0.1899	19.02	3.83	1.46e-04	1.20
2750	5.01	1.7339	0.4277	27.62	5.56	1.27e-04	0.90
3000	5.46	1.8827	0.4273	23.53	4.81	1.07e-04	2.54
3250	5.92	1.3860	0.2982	21.67	4.29	8.81e-05	0.87
3500	6.38	1.1420	0.2716	20.27	4.06	6.88e-05	2.32
3750	6.83	1.0783	0.2688	20.36	4.09	4.96e-05	1.98
4000	7.29	1.0776	0.2657	20.31	4.09	3.03e-05	1.06
4250	7.74	1.0411	0.2602	20.39	4.10	1.10e-05	0.48
4392	8.00		0.2594	20.44	4.11