mms-1b-bigcgen-female-15hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.5205

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.809	0.1003	100	inf	1.0120
6.0699	0.2005	200	inf	0.9993
5.3221	0.3008	300	inf	1.0347
4.0785	0.4010	400	inf	0.7821
2.0743	0.5013	500	inf	0.6082
1.7578	0.6015	600	inf	0.5732
1.8081	0.7018	700	inf	0.5659
1.7107	0.8020	800	inf	0.5478
1.7206	0.9023	900	inf	0.5489
1.6957	1.0020	1000	inf	0.5446
1.587	1.1023	1100	inf	0.5380
1.5794	1.2025	1200	inf	0.5355
1.4728	1.3028	1300	inf	0.5251
1.5137	1.4030	1400	inf	0.5380
1.5073	1.5033	1500	inf	0.5294
1.3676	1.6035	1600	inf	0.5271
1.5592	1.7038	1700	inf	0.5240
1.5091	1.8040	1800	inf	0.5682
1.5439	1.9043	1900	inf	0.5208
1.4025	2.0040	2000	inf	0.5276
1.465	2.1043	2100	inf	0.5269
1.4096	2.2045	2200	inf	0.5346
1.428	2.3048	2300	inf	0.5212
1.3829	2.4050	2400	inf	0.5217
1.3048	2.5053	2500	inf	0.5205

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0