mt5-small-finetuned-amazon-en-kitchen-reviews

This model is a fine-tuned version of google/mt5-small on the amazon_reviews_multi dataset. It achieves the following results on the evaluation set:

• Loss: 1.0960
• Rouge1: 19.1669
• Rouge2: 10.8937
• Rougel: 18.6296
• Rougelsum: 18.7486

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
13.803	1.0	827	2.4239	1.7983	0.0	1.8317	1.7854
1.938	2.0	1654	1.2836	14.9804	4.7997	14.3848	14.4255
1.2922	3.0	2481	1.1718	16.3574	7.2689	15.6126	15.5685
1.1463	4.0	3308	1.1265	17.6554	8.9813	17.1575	17.2073
1.078	5.0	4135	1.1085	19.2978	11.5604	18.8279	18.9399
1.0453	6.0	4962	1.1070	19.3828	11.0161	18.7636	18.9002
1.0233	7.0	5789	1.1004	19.0604	10.5071	18.341	18.4669
1.012	8.0	6616	1.0960	19.1669	10.8937	18.6296	18.7486

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.1