asr-africa/mms-1B_all_DigitalUmuganda_Afrivoice_Shona_10hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the DigitalUmuganda dataset. It achieves the following results on the evaluation set:

• Loss: 0.2319
• Model Preparation Time: 0.0155
• Wer: 0.2828
• Cer: 0.0511

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 150
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
39.2084	0.9955	55	3.1252	0.0155	1.0	1.0
5.8814	1.9910	110	0.2362	0.0155	0.3325	0.0555
1.0775	2.9864	165	0.2061	0.0155	0.3257	0.0531
0.9777	4.0	221	0.1986	0.0155	0.3099	0.0508
0.9527	4.9955	276	0.1949	0.0155	0.3001	0.0490
0.9251	5.9910	331	0.1922	0.0155	0.3017	0.0490
0.9035	6.9864	386	0.1874	0.0155	0.2958	0.0478
0.8627	8.0	442	0.1856	0.0155	0.2860	0.0465
0.8562	8.9955	497	0.1847	0.0155	0.3021	0.0479
0.8375	9.9910	552	0.1819	0.0155	0.2859	0.0465
0.8356	10.9864	607	0.1845	0.0155	0.2906	0.0467
0.8	12.0	663	0.1837	0.0155	0.2914	0.0468
0.8013	12.9955	718	0.1800	0.0155	0.2885	0.0464
0.7971	13.9910	773	0.1808	0.0155	0.2824	0.0453
0.7772	14.9864	828	0.1813	0.0155	0.2828	0.0456
0.753	16.0	884	0.1802	0.0155	0.2841	0.0455
0.7561	16.9955	939	0.1790	0.0155	0.2842	0.0459
0.7417	17.9910	994	0.1789	0.0155	0.2852	0.0452
0.7267	18.9864	1049	0.1789	0.0155	0.2774	0.0447
0.71	20.0	1105	0.1781	0.0155	0.2748	0.0442
0.7071	20.9955	1160	0.1783	0.0155	0.2736	0.0445
0.6976	21.9910	1215	0.1785	0.0155	0.2726	0.0445
0.69	22.9864	1270	0.1775	0.0155	0.2749	0.0441
0.6646	24.0	1326	0.1768	0.0155	0.2747	0.0441
0.6679	24.9955	1381	0.1785	0.0155	0.2687	0.0434
0.6646	25.9910	1436	0.1786	0.0155	0.2745	0.0438
0.6537	26.9864	1491	0.1798	0.0155	0.2744	0.0442
0.63	28.0	1547	0.1792	0.0155	0.2710	0.0439
0.6393	28.9955	1602	0.1792	0.0155	0.2807	0.0441
0.6217	29.9910	1657	0.1773	0.0155	0.2766	0.0442
0.6264	30.9864	1712	0.1802	0.0155	0.2716	0.0433
0.5984	32.0	1768	0.1795	0.0155	0.2728	0.0435
0.6053	32.9955	1823	0.1803	0.0155	0.2714	0.0436
0.6015	33.9910	1878	0.1824	0.0155	0.2663	0.0432
0.591	34.9864	1933	0.1830	0.0155	0.2770	0.0441
0.5688	36.0	1989	0.1807	0.0155	0.2689	0.0432
0.5693	36.9955	2044	0.1854	0.0155	0.2695	0.0433
0.5646	37.9910	2099	0.1827	0.0155	0.2714	0.0435
0.5557	38.9864	2154	0.1824	0.0155	0.2695	0.0434
0.5403	40.0	2210	0.1853	0.0155	0.2720	0.0433
0.5493	40.9955	2265	0.1844	0.0155	0.2724	0.0436
0.5377	41.9910	2320	0.1841	0.0155	0.2675	0.0429
0.5289	42.9864	2375	0.1871	0.0155	0.2674	0.0427
0.5158	44.0	2431	0.1840	0.0155	0.2748	0.0435

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.0.2
• Tokenizers 0.20.1