Hubert-noisy_common_voice_debug

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_NOISY_COMMON_VOICE - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.9787
• Wer: 1.0
• Cer: 0.3145

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
No log	0.2660	100	12.0822	1.1570	1.0539
No log	0.5319	200	5.8789	1.0	0.9817
No log	0.7979	300	5.3627	1.0	0.9817
No log	1.0638	400	4.9316	1.0	0.9817
6.372	1.3298	500	4.4556	1.0	0.9817
6.372	1.5957	600	3.9890	1.0	0.9817
6.372	1.8617	700	3.5734	1.0	0.9817
6.372	2.1277	800	3.2932	1.0	0.9817
6.372	2.3936	900	3.1536	1.0	0.9817
3.4101	2.6596	1000	3.0484	1.0	0.9817
3.4101	2.9255	1100	2.8470	1.0	0.9808
3.4101	3.1915	1200	2.5211	1.0	0.8702
3.4101	3.4574	1300	2.0354	1.0	0.5518
3.4101	3.7234	1400	1.6780	1.0	0.4414
2.3222	3.9894	1500	1.5039	1.0	0.4312
2.3222	4.2553	1600	1.3419	1.0	0.3965
2.3222	4.5213	1700	1.2054	1.0	0.3686
2.3222	4.7872	1800	1.0588	1.0	0.3321
2.3222	5.0532	1900	0.9546	1.0	0.3158
1.2343	5.3191	2000	0.9042	1.0	0.3106
1.2343	5.5851	2100	0.8747	1.0	0.3088
1.2343	5.8511	2200	0.8224	1.0	0.2972
1.2343	6.1170	2300	0.8101	1.0	0.2996
1.2343	6.3830	2400	0.7892	1.0	0.2970
0.8716	6.6489	2500	0.7661	1.0	0.2915
0.8716	6.9149	2600	0.7654	1.0	0.2886
0.8716	7.1809	2700	0.7677	1.0	0.2898
0.8716	7.4468	2800	0.7528	1.0	0.2861
0.8716	7.7128	2900	0.7433	1.0	0.2880
0.7324	7.9787	3000	0.7498	1.0	0.2877
0.7324	8.2447	3100	0.7267	1.0	0.2827
0.7324	8.5106	3200	0.7319	1.0	0.2813
0.7324	8.7766	3300	0.7478	1.0	0.2882
0.7324	9.0426	3400	0.7337	1.0	0.2815
0.6486	9.3085	3500	0.7341	1.0	0.2851
0.6486	9.5745	3600	0.7419	1.0	0.2803
0.6486	9.8404	3700	0.7033	0.9998	0.2773
0.6486	10.1064	3800	0.7327	1.0	0.2829
0.6486	10.3723	3900	0.7554	0.9998	0.2855
0.6034	10.6383	4000	0.7361	1.0	0.2841
0.6034	10.9043	4100	0.7459	1.0	0.2833
0.6034	11.1702	4200	0.7384	1.0	0.2801
0.6034	11.4362	4300	0.7337	1.0	0.2776
0.6034	11.7021	4400	0.7572	1.0	0.2819
0.5687	11.9681	4500	0.7522	1.0	0.2824
0.5687	12.2340	4600	0.7491	1.0	0.2789
0.5687	12.5	4700	0.7485	1.0	0.2832
0.5687	12.7660	4800	0.7623	1.0	0.2849
0.5687	13.0319	4900	0.7829	1.0	0.2859
0.5255	13.2979	5000	0.7819	1.0	0.2820
0.5255	13.5638	5100	0.7783	0.9998	0.2824
0.5255	13.8298	5200	0.7653	1.0	0.2840
0.5255	14.0957	5300	0.7816	1.0	0.2822
0.5255	14.3617	5400	0.7608	1.0	0.2824
0.5016	14.6277	5500	0.7712	0.9998	0.2841
0.5016	14.8936	5600	0.7712	1.0	0.2864
0.5016	15.1596	5700	0.8153	0.9996	0.2851
0.5016	15.4255	5800	0.8161	0.9998	0.2852
0.5016	15.6915	5900	0.7911	1.0	0.2883
0.4821	15.9574	6000	0.7926	1.0	0.2823
0.4821	16.2234	6100	0.8147	1.0	0.2867
0.4821	16.4894	6200	0.7700	1.0	0.2826
0.4821	16.7553	6300	0.8119	1.0	0.2910
0.4821	17.0213	6400	0.8355	1.0	0.2846
0.4503	17.2872	6500	0.7936	0.9998	0.2859
0.4503	17.5532	6600	0.7976	0.9998	0.2952
0.4503	17.8191	6700	0.8274	0.9998	0.2902
0.4503	18.0851	6800	0.9034	0.9998	0.2885
0.4503	18.3511	6900	0.8066	0.9998	0.2882
0.4435	18.6170	7000	0.8495	1.0	0.2921
0.4435	18.8830	7100	0.8448	0.9998	0.2896
0.4435	19.1489	7200	0.8774	1.0	0.2904
0.4435	19.4149	7300	0.8293	0.9998	0.2973
0.4435	19.6809	7400	0.8038	1.0	0.2925
0.4457	19.9468	7500	0.8062	0.9998	0.2908
0.4457	20.2128	7600	0.8740	1.0	0.2918
0.4457	20.4787	7700	0.8489	1.0	0.2977
0.4457	20.7447	7800	0.8606	1.0	0.2973
0.4457	21.0106	7900	0.8141	0.9998	0.2926
0.4252	21.2766	8000	0.8832	0.9998	0.2984
0.4252	21.5426	8100	0.8590	0.9998	0.2945
0.4252	21.8085	8200	0.8304	0.9998	0.2940
0.4252	22.0745	8300	0.8734	0.9998	0.2974
0.4252	22.3404	8400	0.8417	0.9998	0.2930
0.418	22.6064	8500	0.9387	1.0	0.2993
0.418	22.8723	8600	0.8810	1.0	0.2996
0.418	23.1383	8700	0.9090	1.0	0.3074
0.418	23.4043	8800	0.8993	0.9998	0.3107
0.418	23.6702	8900	0.8724	1.0	0.3033
0.424	23.9362	9000	0.8895	0.9998	0.3042
0.424	24.2021	9100	0.8863	1.0	0.3014
0.424	24.4681	9200	0.9255	0.9998	0.3112
0.424	24.7340	9300	0.9398	0.9998	0.3011
0.424	25.0	9400	0.8763	0.9998	0.3071
0.4122	25.2660	9500	0.9353	1.0	0.3090
0.4122	25.5319	9600	0.9382	1.0	0.3128
0.4122	25.7979	9700	0.9295	0.9998	0.3102
0.4122	26.0638	9800	0.9286	0.9998	0.3092
0.4122	26.3298	9900	0.9141	1.0	0.3014
0.4146	26.5957	10000	0.9426	1.0	0.3126
0.4146	26.8617	10100	0.8652	1.0	0.3032
0.4146	27.1277	10200	0.9289	1.0	0.3105
0.4146	27.3936	10300	0.9459	1.0	0.3103
0.4146	27.6596	10400	0.9137	0.9998	0.3124
0.416	27.9255	10500	0.9305	1.0	0.3100
0.416	28.1915	10600	0.9589	1.0	0.3071
0.416	28.4574	10700	0.9276	0.9998	0.3061
0.416	28.7234	10800	0.9259	1.0	0.3078
0.416	28.9894	10900	0.9287	1.0	0.3150
0.4078	29.2553	11000	0.9346	0.9998	0.3166
0.4078	29.5213	11100	0.9559	1.0	0.3165
0.4078	29.7872	11200	0.9670	1.0	0.3173

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3