xlsr-1b-mizonal3-E5-lus-v2026.06

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.

It achieves the following results on the evaluation set:

• Wer: 26.3669
• Cer: 5.7509
• Real Time Factor: 0.0038

Quick Inference

import torch
import librosa
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2Processor.from_pretrained("andrewbawitlung/xlsr-1b-mizonal3-E5-lus-v2026.06")
model = Wav2Vec2ForCTC.from_pretrained("andrewbawitlung/xlsr-1b-mizonal3-E5-lus-v2026.06").to(device)

audio, sr = librosa.load("your_audio.wav", sr=16000)
input_values = processor(audio, sampling_rate=16000, return_tensors="pt").input_values.to(device)

with torch.no_grad():
    logits = model(input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)

Model description

Experiment Configurations

This repository is part of a series of experiments. The different configurations are:

• E1 (Baseline): Standard training configuration.
• E2 (Noise): Training with background noise augmentation.
• E3 (Speed): Training with speed perturbation augmentation.
• E4 (SpecAug): Training with SpecAugment (time and frequency masking).
• E5 (Combined): Training with a combination of all augmentations.

All Models in this Family

Experiment	Hugging Face Repository
E1 (Baseline)	andrewbawitlung/xlsr-1b-mizonal3-E1-lus-v2026.06
E2 (Noise)	andrewbawitlung/xlsr-1b-mizonal3-E2-lus-v2026.06
E3 (Speed)	andrewbawitlung/xlsr-1b-mizonal3-E3-lus-v2026.06
E4 (SpecAug)	andrewbawitlung/xlsr-1b-mizonal3-E4-lus-v2026.06
E5 (Combined)	andrewbawitlung/xlsr-1b-mizonal3-E5-lus-v2026.06

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• lr_scheduler_type: SchedulerType.LINEAR
• num_epochs: 8

Training results

step	epoch	train_loss	eval_loss	eval_wer	eval_cer	learning_rate	grad_norm
250	0.23	1.1180	0.4791	51.50	12.84	1.49e-04	1.25
500	0.46	0.8414	0.3856	42.35	10.66	2.99e-04	1.06
750	0.68	0.6714	0.3248	38.09	9.10	2.91e-04
1000	0.91	0.5910	0.2621	31.19	7.28	2.82e-04	1.04
1250	1.14	0.5168	0.2403	28.85	6.56	2.73e-04	0.89
1500	1.37	0.4791	0.2242	27.38	6.17	2.64e-04	0.70
1750	1.59	0.4679	0.2072	25.12	5.49	2.55e-04	0.68
2000	1.82	0.4817	0.2280	28.10	6.03	2.46e-04	0.75
2250	2.05	1.0189	0.9858	84.89	35.61	2.37e-04	3.39
2500	2.28	1.4424	1.2154	87.59	41.69	2.28e-04	2.31
2750	2.50	1.2047	0.9387	67.68	23.83	2.19e-04	1.51
3000	2.73	1.0050	0.7155	42.30	11.26	2.10e-04	1.51
3250	2.96	0.8264	0.5677	33.68	7.93	2.00e-04	1.44
3500	3.19	0.7819	0.4931	31.72	6.99	1.91e-04	1.90
3750	3.42	0.7419	0.4560	31.37	6.94	1.82e-04	2.07
4000	3.64	0.7366	0.4494	30.98	6.79	1.73e-04	1.51
4250	3.87	0.7231	0.4534	30.60	6.74	1.64e-04	1.21
4500	4.10	0.7773	0.4591	33.36	7.53	1.55e-04	0.42
4750	4.33	0.7846	0.4708	35.42	7.63	1.46e-04	0.00
5000	4.55	0.7707	0.4708	35.40	7.63	1.37e-04	0.00
5250	4.78	0.7770	0.4708	35.40	7.63	1.28e-04	0.00
5500	5.01	0.7748	0.4708	35.40	7.63	1.19e-04	0.00
5750	5.24	0.7671	0.4708	35.40	7.63	1.10e-04	0.00
6000	5.46	0.7716	0.4708	35.40	7.63	1.01e-04	0.00
6250	5.69	0.7678	0.4708	35.40	7.63	9.18e-05	0.00
6500	5.92	0.7738	0.4708	35.40	7.63	8.27e-05	0.00
6750	6.15	0.7774	0.4708	35.40	7.63	7.37e-05	0.00
7000	6.38	0.7625	0.4708	35.40	7.63	6.46e-05	0.00
7250	6.60	0.7610	0.4708	35.40	7.63	5.56e-05	0.00
7500	6.83	0.7590	0.4708	35.40	7.63	4.65e-05	0.00
7750	7.06	0.7725	0.4708	35.40	7.63	3.75e-05	0.00
8000	7.29	0.7572	0.4708	35.40	7.63	2.84e-05	0.00
8250	7.51	0.7719	0.4708	35.40	7.63	1.94e-05	0.00
8500	7.74	0.7734	0.4708	35.40	7.63	1.03e-05	0.00
8750	7.97	0.7651	0.4708	35.40	7.63	1.27e-06	0.00