asr-africa/mms-1B_all_DigitalUmuganda_Afrivoice_Shona_20hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the DigitalUmuganda dataset. It achieves the following results on the evaluation set:

• Loss: 0.2338
• Model Preparation Time: 0.0115
• Wer: 0.2764
• Cer: 0.0510

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 150
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
36.4063	0.9954	109	5.1803	0.0115	1.0	0.9463
19.0038	2.0	219	4.0762	0.0115	1.0696	0.7878
14.6479	2.9954	328	3.2461	0.0115	0.9999	0.9420
12.4828	4.0	438	2.8917	0.0115	1.0	0.8262
11.4667	4.9954	547	2.7706	0.0115	1.0006	0.7752
10.9805	6.0	657	2.6967	0.0115	1.0007	0.7594
10.6692	6.9954	766	2.4726	0.0115	1.0	0.7709
2.4538	8.0	876	0.2426	0.0115	0.3420	0.0586
1.0883	8.9954	985	0.2242	0.0115	0.3166	0.0541
1.0062	10.0	1095	0.2119	0.0115	0.3104	0.0523
0.9739	10.9954	1204	0.2072	0.0115	0.3081	0.0510
0.934	12.0	1314	0.2035	0.0115	0.2992	0.0498
0.9141	12.9954	1423	0.2000	0.0115	0.2934	0.0492
0.8906	14.0	1533	0.1973	0.0115	0.2896	0.0485
0.8799	14.9954	1642	0.1958	0.0115	0.2883	0.0478
0.8563	16.0	1752	0.1952	0.0115	0.2843	0.0475
0.8487	16.9954	1861	0.1929	0.0115	0.2841	0.0469
0.8292	18.0	1971	0.1909	0.0115	0.2799	0.0466
0.8203	18.9954	2080	0.1892	0.0115	0.2815	0.0468
0.8039	20.0	2190	0.1897	0.0115	0.2789	0.0460
0.8014	20.9954	2299	0.1899	0.0115	0.2852	0.0470
0.7822	22.0	2409	0.1883	0.0115	0.2761	0.0457
0.783	22.9954	2518	0.1860	0.0115	0.2757	0.0456
0.765	24.0	2628	0.1869	0.0115	0.2728	0.0451
0.7608	24.9954	2737	0.1844	0.0115	0.2701	0.0448
0.7494	26.0	2847	0.1851	0.0115	0.2704	0.0444
0.7414	26.9954	2956	0.1841	0.0115	0.2713	0.0450
0.73	28.0	3066	0.1855	0.0115	0.2672	0.0443
0.7282	28.9954	3175	0.1859	0.0115	0.2706	0.0442
0.7164	30.0	3285	0.1837	0.0115	0.2683	0.0441
0.7145	30.9954	3394	0.1835	0.0115	0.2743	0.0447
0.7027	32.0	3504	0.1855	0.0115	0.2739	0.0443
0.6973	32.9954	3613	0.1843	0.0115	0.2674	0.0438
0.6869	34.0	3723	0.1832	0.0115	0.2699	0.0440
0.683	34.9954	3832	0.1834	0.0115	0.2722	0.0441
0.6677	36.0	3942	0.1818	0.0115	0.2684	0.0439
0.67	36.9954	4051	0.1825	0.0115	0.2631	0.0435
0.6577	38.0	4161	0.1850	0.0115	0.2744	0.0442
0.6545	38.9954	4270	0.1819	0.0115	0.2694	0.0438
0.6412	40.0	4380	0.1860	0.0115	0.2711	0.0438
0.6455	40.9954	4489	0.1837	0.0115	0.2635	0.0431
0.6259	42.0	4599	0.1841	0.0115	0.2638	0.0430
0.6344	42.9954	4708	0.1855	0.0115	0.2671	0.0434
0.619	44.0	4818	0.1840	0.0115	0.2627	0.0428
0.6137	44.9954	4927	0.1857	0.0115	0.2596	0.0426
0.6058	46.0	5037	0.1827	0.0115	0.2656	0.0430
0.6009	46.9954	5146	0.1851	0.0115	0.2654	0.0429
0.5885	48.0	5256	0.1837	0.0115	0.2619	0.0423
0.5892	48.9954	5365	0.1851	0.0115	0.2681	0.0429
0.5741	50.0	5475	0.1869	0.0115	0.2610	0.0427
0.5812	50.9954	5584	0.1861	0.0115	0.2600	0.0426
0.5653	52.0	5694	0.1847	0.0115	0.2651	0.0431
0.5604	52.9954	5803	0.1886	0.0115	0.2641	0.0428
0.5532	54.0	5913	0.1891	0.0115	0.2638	0.0428
0.5533	54.9954	6022	0.1894	0.0115	0.2602	0.0421

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.0.2
• Tokenizers 0.20.1