wav2vec2-large-xls-r-1b-cv-13-elderly-frisian

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on a subset of the Common Voice dataset for the elderly population. It achieves the following results on the evaluation set:

• Loss: 0.6129
• Wer: 0.4703

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 80
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.2575	5.0	50	3.0613	1.0
3.0917	10.0	100	3.1951	1.0
2.9905	15.0	150	2.8278	1.0
2.6373	20.0	200	1.4290	1.0013
1.8438	25.0	250	0.8604	0.7939
1.1253	30.0	300	0.7872	0.7009
0.7395	35.0	350	0.6323	0.5780
0.7408	40.0	400	0.6096	0.5315
0.8041	45.0	450	0.6081	0.5239
0.6946	50.0	500	0.6098	0.5074
0.5954	55.0	550	0.6358	0.5132
0.5571	60.0	600	0.6034	0.4922
0.4828	65.0	650	0.6084	0.4774
0.6639	70.0	700	0.5984	0.4712
0.5682	75.0	750	0.5985	0.4712
0.3801	80.0	800	0.6129	0.4703

Framework versions

• Transformers 4.27.4
• Pytorch 2.0.0+cu117
• Datasets 2.11.0
• Tokenizers 0.13.2