pakawadeep/mt5-base-finetuned-ctfl-augmented_05

This model is a fine-tuned version of google/mt5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.3668
• Validation Loss: 0.7815
• Train Rouge1: 8.5926
• Train Rouge2: 0.7921
• Train Rougel: 8.5573
• Train Rougelsum: 8.7341
• Train Gen Len: 11.8861
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Train Gen Len	Epoch
5.9893	2.6002	4.4884	0.4950	4.3454	4.3399	8.3168	0
2.9777	2.0796	4.7619	0.3300	4.7000	4.8444	10.2723	1
2.3190	1.7755	6.4356	1.0891	6.5064	6.5535	10.6881	2
1.8959	1.4532	7.4965	1.5842	7.4257	7.5436	11.3911	3
1.5823	1.2455	7.7086	2.0792	7.4965	7.7086	11.7673	4
1.3569	1.1076	8.4866	2.0792	8.4866	8.6987	11.8861	5
1.2095	1.0615	8.4866	2.0792	8.4866	8.6987	11.9802	6
1.1007	0.9902	8.9109	2.2772	8.7694	8.9816	11.9455	7
1.0050	0.9564	8.9109	2.2772	8.7694	8.9816	11.9455	8
0.9284	0.8988	8.6987	1.7822	8.6752	8.8166	11.9307	9
0.8636	0.8796	8.6987	1.7822	8.6752	8.8166	11.9109	10
0.8182	0.8624	8.3687	1.2871	8.3628	8.5337	11.8762	11
0.7744	0.8172	8.3687	1.2871	8.3628	8.5337	11.8812	12
0.7333	0.8051	8.3687	1.2871	8.3628	8.5337	11.9208	13
0.6928	0.8167	8.2744	1.2871	8.2744	8.4689	11.9109	14
0.6587	0.7863	8.2744	1.2871	8.2744	8.4689	11.8960	15
0.6302	0.7844	8.2744	1.2871	8.2744	8.4689	11.8960	16
0.6025	0.7772	8.2744	1.2871	8.2744	8.4689	11.9059	17
0.5816	0.7697	8.2744	1.2871	8.2744	8.4689	11.8861	18
0.5561	0.7654	8.2744	1.2871	8.2744	8.4689	11.8317	19
0.5359	0.7703	8.2744	1.2871	8.2744	8.4689	11.8861	20
0.5144	0.7546	7.9208	0.7921	7.9444	8.0387	11.8564	21
0.4976	0.7550	7.9208	0.7921	7.9444	8.0387	11.8614	22
0.4725	0.7612	7.9208	0.7921	7.9444	8.0387	11.8515	23
0.4520	0.7772	7.9208	0.7921	7.9444	8.0387	11.8218	24
0.4321	0.7647	7.9208	0.7921	7.9444	8.0387	11.8069	25
0.4157	0.7598	8.4866	0.7921	8.4335	8.5573	11.8218	26
0.3985	0.7657	8.7694	1.2871	8.7694	8.9816	11.8812	27
0.3863	0.7613	8.4158	0.7921	8.4158	8.4689	11.8218	28
0.3668	0.7815	8.5926	0.7921	8.5573	8.7341	11.8861	29

Framework versions

• Transformers 4.41.2
• TensorFlow 2.15.0
• Datasets 2.20.0
• Tokenizers 0.19.1