pakawadeep/mt5-base-finetuned-ctfl-augmented

This model is a fine-tuned version of google/mt5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.4325
• Validation Loss: 0.7749
• Train Rouge1: 8.6103
• Train Rouge2: 0.7921
• Train Rougel: 8.6987
• Train Rougelsum: 8.6987
• Train Gen Len: 11.8713
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Train Gen Len	Epoch
6.9468	2.5770	1.3751	0.2405	1.3339	1.3916	8.5050	0
3.5520	2.0970	5.5693	0.8251	5.5487	5.6106	10.8564	1
2.7376	2.2001	5.1568	1.3201	4.9505	5.1155	10.0099	2
2.4757	1.8936	6.2706	1.1881	6.2235	6.3885	10.3614	3
2.1064	1.5432	7.4045	1.6832	7.2136	7.4116	11.1040	4
1.8167	1.3532	8.4866	2.1782	8.4158	8.6987	11.5644	5
1.6032	1.1789	8.6987	2.1782	8.4866	8.6987	11.8267	6
1.4351	1.1083	8.6987	2.1782	8.4866	8.6987	11.9059	7
1.3021	1.0607	8.9109	2.3762	8.8048	8.9816	11.9604	8
1.2060	1.0120	8.9109	2.3762	8.8048	8.9816	11.9455	9
1.1182	0.9736	8.6987	1.8812	8.6987	8.7694	11.9703	10
1.0551	0.9458	8.6987	1.8812	8.6987	8.7694	11.9406	11
0.9862	0.9170	8.5926	1.3861	8.5337	8.6516	11.9455	12
0.9324	0.8727	8.5101	1.3861	8.4807	8.5691	11.9208	13
0.8809	0.8625	8.7459	1.8812	8.6987	8.8166	11.9257	14
0.8334	0.8322	8.4512	1.3861	8.4158	8.4866	11.9307	15
0.7864	0.8009	7.9562	0.7921	7.7793	7.9915	11.9010	16
0.7542	0.8049	8.2744	0.7921	8.2390	8.3215	11.8465	17
0.7157	0.7914	8.4512	1.3861	8.4158	8.4866	11.9356	18
0.6806	0.7787	8.2744	0.7921	8.2390	8.3215	11.8812	19
0.6492	0.7813	8.2744	0.7921	8.2390	8.3215	11.8861	20
0.6222	0.7654	8.2744	0.7921	8.2390	8.3215	11.8713	21
0.5934	0.7588	8.4512	1.3861	8.4158	8.4866	11.8960	22
0.5648	0.7596	8.2744	0.7921	8.2390	8.3215	11.8861	23
0.5423	0.7755	8.2744	0.7921	8.2390	8.3215	11.8762	24
0.5180	0.7640	8.2744	0.7921	8.2390	8.3215	11.8861	25
0.4931	0.7582	8.2744	0.7921	8.2390	8.3215	11.8416	26
0.4738	0.7671	8.2744	0.7921	8.2390	8.3215	11.8614	27
0.4544	0.7644	8.2744	0.7921	8.2390	8.3215	11.8465	28
0.4325	0.7749	8.6103	0.7921	8.6987	8.6987	11.8713	29

Framework versions

• Transformers 4.38.2
• TensorFlow 2.15.0
• Datasets 2.18.0
• Tokenizers 0.15.2