mms-1b-bigcgen-male-25hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.4361
• Wer: 0.4427

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.7421	0.0623	100	0.9477	0.8055
1.8228	0.1245	200	0.5965	0.5665
1.5859	0.1868	300	0.5951	0.5415
1.4482	0.2491	400	0.5432	0.5314
1.3757	0.3113	500	0.5301	0.5186
1.4261	0.3736	600	0.5236	0.5335
1.3224	0.4359	700	0.5211	0.5114
1.495	0.4981	800	0.5083	0.5047
1.277	0.5604	900	0.4756	0.4891
1.1762	0.6227	1000	0.4588	0.4804
1.1687	0.6849	1100	0.4574	0.4828
1.2752	0.7472	1200	0.4439	0.4732
1.3276	0.8095	1300	0.4488	0.4770
1.1854	0.8717	1400	0.4453	0.4727
1.1814	0.9340	1500	0.4435	0.4612
1.1269	0.9963	1600	0.4503	0.4638
1.1957	1.0585	1700	0.4407	0.4631
1.1366	1.1208	1800	0.4379	0.4552
1.2092	1.1831	1900	0.4385	0.4629
1.1523	1.2453	2000	0.4447	0.4564
1.0315	1.3076	2100	0.4424	0.4489
1.1741	1.3699	2200	0.4341	0.4528
1.1343	1.4321	2300	0.4447	0.4540
1.1826	1.4944	2400	0.4353	0.4395
1.1368	1.5567	2500	0.4359	0.4491
1.2155	1.6189	2600	0.4361	0.4429

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0