DeCRED_small_cv

This model is a fine-tuned version of on the common_voice_13_0 dataset. It achieves the following results on the evaluation set:

• Cer: 0.0855
• Deletions: 4133
• Hits: 123172
• Insertions: 3671
• Loss: 1.1174
• Mer: 0.1889
• Substitutions: 20876
• Wer: 0.1935
• Wil: 0.3069
• Wip: 0.6931

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.002
• train_batch_size: 256
• eval_batch_size: 128
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 512
• total_eval_batch_size: 256
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10000
• num_epochs: 20.0

Training results

Training Loss	Epoch	Step	Cer	Deletions	Hits	Insertions	Validation Loss	Mer	Substitutions	Wer	Wil	Wip
1.3459	5.0	7445	0.1676	7277	102705	6830	1.5193	0.3374	38199	0.3530	0.5182	0.4818
1.3006	6.0	8934	0.1584	6381	105485	7074	1.4779	0.3206	36315	0.3359	0.4956	0.5044
1.7549	7.0	10423	0.1620	6967	104307	6557	1.4874	0.3259	36907	0.3403	0.5031	0.4969
1.5621	8.0	11912	0.1361	5203	111002	6489	1.3687	0.2823	31976	0.2947	0.4437	0.5563
1.4988	9.0	13401	0.1298	5025	112303	6136	1.3308	0.2723	30853	0.2835	0.4299	0.5701
1.4495	10.0	14890	0.1212	5505	113957	5063	1.2901	0.2564	28719	0.2651	0.4068	0.5932
1.4092	11.0	16379	0.1158	5162	115248	4910	1.2625	0.2472	27771	0.2554	0.3941	0.6059
1.3638	12.0	17868	0.1088	4408	117314	5173	1.2336	0.2350	26459	0.2432	0.3764	0.6236
1.335	13.0	19357	0.1060	4868	117783	4467	1.2177	0.2284	25530	0.2353	0.3665	0.6335
1.2966	14.0	20846	0.1006	4561	119385	4599	1.1945	0.2186	24235	0.2254	0.3511	0.6489
1.2826	15.0	22335	0.0969	4455	119983	4111	1.1767	0.2122	23743	0.2180	0.3429	0.6571
1.2529	16.0	23824	0.0954	4229	120627	4312	1.1664	0.2090	23325	0.2150	0.3377	0.6623
1.2103	17.0	25313	0.0907	4218	121711	3998	1.1458	0.2002	22252	0.2056	0.3244	0.6756
1.18	18.0	26802	0.0891	4153	122139	3883	1.1355	0.1968	21889	0.2019	0.3194	0.6806
1.1529	19.0	28291	0.0864	4132	122860	3681	1.1236	0.1910	21189	0.1957	0.3105	0.6895
1.1342	20.0	29780	0.0855	4133	123172	3671	1.1174	0.1889	20876	0.1935	0.3069	0.6931

Framework versions

• Transformers 4.40.0.dev0
• Pytorch 2.2.0+rocm5.6
• Datasets 2.18.0
• Tokenizers 0.15.2

Wandb run

https://wandb.ai/butspeechfit/decred_commonvoice_en/runs/DeCRED_small_cv_restart