wav2vec2-xls-r-300m-en-phoneme-ctc-60h-balanced

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1532
• Per: 0.0378
• Phoneme Accuracy: 0.9622

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Per	Phoneme Accuracy
4.9502	0.3185	500	4.3875	1.0	0.0
3.6506	0.6369	1000	3.5930	1.0	0.0
3.6533	0.9554	1500	3.5748	1.0	0.0
3.6035	1.2739	2000	3.5249	1.0	0.0
1.4335	1.5924	2500	0.7984	0.1635	0.8365
0.6011	1.9108	3000	0.3071	0.0706	0.9294
0.4718	2.2293	3500	0.2315	0.0576	0.9424
0.4498	2.5478	4000	0.2091	0.0528	0.9472
0.3683	2.8662	4500	0.1897	0.0504	0.9496
0.331	3.1847	5000	0.1810	0.0472	0.9528
0.3256	3.5032	5500	0.1725	0.0458	0.9542
0.2949	3.8217	6000	0.1643	0.0441	0.9559
0.2894	4.1401	6500	0.1662	0.0436	0.9564
0.2542	4.4586	7000	0.1592	0.0430	0.9570
0.2422	4.7771	7500	0.1563	0.0433	0.9567
0.2485	5.0955	8000	0.1577	0.0417	0.9583
0.2592	5.4140	8500	0.1538	0.0427	0.9573
0.2295	5.7325	9000	0.1520	0.0412	0.9588
0.215	6.0510	9500	0.1493	0.0402	0.9598
0.2304	6.3694	10000	0.1508	0.0410	0.9590
0.2111	6.6879	10500	0.1475	0.0397	0.9603
0.2007	7.0064	11000	0.1477	0.0393	0.9607
0.1947	7.3248	11500	0.1529	0.0390	0.9610
0.1895	7.6433	12000	0.1512	0.0390	0.9610
0.2178	7.9618	12500	0.1443	0.0396	0.9604
0.2088	8.2803	13000	0.1467	0.0393	0.9607
0.1813	8.5987	13500	0.1488	0.0383	0.9617
0.1796	8.9172	14000	0.1450	0.0384	0.9616
0.1657	9.2357	14500	0.1474	0.0382	0.9618
0.1687	9.5541	15000	0.1478	0.0384	0.9616
0.1759	9.8726	15500	0.1452	0.0379	0.9621
0.1699	10.1911	16000	0.1495	0.0376	0.9624
0.1737	10.5096	16500	0.1492	0.0374	0.9626
0.1784	10.8280	17000	0.1465	0.0385	0.9615
0.1697	11.1465	17500	0.1480	0.0385	0.9615
0.1618	11.4650	18000	0.1477	0.0375	0.9625
0.1499	11.7834	18500	0.1474	0.0398	0.9602
0.1363	12.1019	19000	0.1532	0.0378	0.9622

Framework versions

• Transformers 4.57.6
• Pytorch 2.11.0+cu130
• Datasets 5.0.0
• Tokenizers 0.22.2