mms-1b-bigcgen-male-30hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.4418
• Wer: 0.4513

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.8177	0.0518	100	0.9883	0.8360
1.9066	0.1035	200	0.6111	0.5658
1.6259	0.1553	300	0.5919	0.5600
1.5344	0.2071	400	0.5518	0.5453
1.5788	0.2589	500	0.5322	0.5417
1.3816	0.3106	600	0.4890	0.5234
1.3242	0.3624	700	0.4798	0.5032
1.301	0.4142	800	0.4813	0.5160
1.1288	0.4660	900	0.4632	0.4895
1.2779	0.5177	1000	0.4606	0.4855
1.172	0.5695	1100	0.4532	0.4835
1.2388	0.6213	1200	0.4610	0.4833
1.2582	0.6731	1300	0.4502	0.4667
1.1046	0.7248	1400	0.4608	0.4696
1.2732	0.7766	1500	0.4450	0.4696
1.2323	0.8284	1600	0.4495	0.4631
1.282	0.8801	1700	0.4401	0.4604
1.1099	0.9319	1800	0.4440	0.4612
1.1463	0.9837	1900	0.4417	0.4689
1.12	1.0352	2000	0.4408	0.4549
1.1037	1.0870	2100	0.4418	0.4513

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0