mms-1B_all_DigitalUmuganda_Afrivoice_Fleurs_Shona_100hr_v1

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1804
• Wer: 0.2279
• Cer: 0.0401

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
13.7336	0.9998	1112	0.8692	2.7700	1.0050
3.7041	1.9996	2224	0.0514	0.2123	0.3067
0.8566	2.9993	3336	0.0473	0.1980	0.2808
0.8086	4.0	4449	0.0472	0.1920	0.2891
0.7772	4.9998	5561	0.0448	0.1855	0.2668
0.7569	5.9996	6673	0.0436	0.1821	0.2622
0.7374	6.9993	7785	0.0427	0.1782	0.2594
0.1774	7.9998	8896	0.1764	0.2531	0.0420
0.1776	8.9996	10008	0.1768	0.2547	0.0421
0.1767	9.9993	11120	0.1765	0.2569	0.0427
0.1758	11.0	12233	0.1752	0.2581	0.0428
0.1736	11.9998	13345	0.1734	0.2529	0.0421
0.1719	12.9996	14457	0.1733	0.2538	0.0421
0.1708	13.9993	15569	0.1721	0.2558	0.0418
0.17	15.0	16682	0.1734	0.2494	0.0415
0.1679	15.9998	17794	0.1705	0.2489	0.0413
0.1669	16.9996	18906	0.1725	0.2483	0.0409
0.1648	17.9993	20018	0.1711	0.2444	0.0405
0.1633	19.0	21131	0.1707	0.2490	0.0407
0.1622	19.9998	22243	0.1704	0.2450	0.0406
0.1609	20.9996	23355	0.1713	0.2485	0.0410
0.1601	21.9993	24467	0.1727	0.2509	0.0412
0.1581	23.0	25580	0.1717	0.2485	0.0404
0.1579	23.9998	26692	0.1699	0.2454	0.0401
0.1562	24.9996	27804	0.1681	0.2444	0.0403
0.155	25.9993	28916	0.1707	0.2446	0.0402
0.1535	27.0	30029	0.1694	0.2450	0.0400
0.1518	27.9998	31141	0.1706	0.2472	0.0406

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.0.2
• Tokenizers 0.20.1