xls-r-1b-bigcgen-combined-25hrs-model

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the BIGCGEN - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.5883
• Wer: 0.6174

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30.0

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.0844	100	3.6907	1.0
No log	0.1687	200	2.8530	1.0
No log	0.2531	300	1.5852	1.0
No log	0.3374	400	0.8783	0.8756
6.0923	0.4218	500	0.7474	0.7954
6.0923	0.5061	600	0.7434	0.6484
6.0923	0.5905	700	0.6424	0.6793
6.0923	0.6748	800	0.6096	0.6079
6.0923	0.7592	900	0.6087	0.6473
1.3442	0.8435	1000	0.5863	0.5745
1.3442	0.9279	1100	0.6088	0.6133
1.3442	1.0118	1200	0.6080	0.6529
1.3442	1.0962	1300	0.5635	0.5794
1.3442	1.1805	1400	0.5650	0.5942
1.2025	1.2649	1500	0.5993	0.6317
1.2025	1.3492	1600	0.5956	0.6487
1.2025	1.4336	1700	0.5566	0.5913
1.2025	1.5179	1800	0.5583	0.5704
1.2025	1.6023	1900	0.5911	0.6397
1.0909	1.6866	2000	0.5841	0.5938
1.0909	1.7710	2100	0.5883	0.6179

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0