xlsr300m-khmer-cpt-10h

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Contrastive Loss: 534.9407
• Diversity Loss: 242.5427
• Codevector Perplexity: 122.9978
• Loss: 559.1949

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.98) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: polynomial
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 50000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Contrastive Loss	Diversity Loss	Codevector Perplexity	Validation Loss
1552.4386	40.8163	1000	594.2692	240.2464	117.9638	618.2938
1423.907	81.6327	2000	572.7025	237.1678	118.7327	596.4193
1362.7096	122.4490	3000	561.6609	236.6906	118.2798	585.3299
1316.401	163.2653	4000	593.5982	242.4052	120.8062	617.8388
1283.7859	204.0816	5000	582.1304	241.7541	120.9417	606.3058
1257.0409	244.8980	6000	555.2018	236.5953	118.5476	578.8614
1233.6443	285.7143	7000	574.9687	243.7643	121.0316	599.3452
1213.2797	326.5306	8000	550.4002	237.9146	118.7674	574.1916
1199.536	367.3469	9000	555.6633	246.2235	120.6750	580.2856
1186.7934	408.1633	10000	562.2374	239.8864	120.3484	586.2260
1172.7632	448.9796	11000	562.9417	245.8130	123.0319	587.5230
1157.9899	489.7959	12000	550.1823	238.6012	117.3512	574.0424
1154.7736	530.6122	13000	534.1852	237.8220	122.0059	557.9674
1137.8447	571.4286	14000	533.3675	235.8615	120.6676	556.9536
1130.5336	612.2449	15000	536.2080	241.8619	121.7849	560.3942
1124.505	653.0612	16000	538.8758	240.9151	118.9648	562.9673
1123.314	693.8776	17000	543.0479	236.3416	120.2038	566.6821
1110.8329	734.6939	18000	557.6065	242.5936	123.2768	581.8659
1102.711	775.5102	19000	534.9407	242.5427	122.9978	559.1949

Framework versions

• Transformers 4.46.3
• Pytorch 2.12.1+cu130
• Datasets 5.0.0
• Tokenizers 0.20.3