outputs-project-id2223

This model is a fine-tuned version of t5-small on the xsum dataset. It achieves the following results on the evaluation set:

• Gen Len: 19.7143
• Loss: 2.2606
• Rouge1: 31.3824
• Rouge2: 9.9473
• Rougel: 24.9422
• Rougelsum: 24.9453

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Gen Len	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.6596	1.0	12753	19.6937	2.4320	29.2361	8.3218	22.9893	22.9993
2.5914	2.0	25506	19.7058	2.3844	29.8093	8.7641	23.5074	23.5169
2.5171	4.0	51012	19.6821	2.3208	30.6455	9.3612	24.2744	24.2798
2.4813	6.0	76518	19.6813	2.2865	31.1312	9.7512	24.6686	24.6688
2.4517	7.0	89271	19.7118	2.2757	31.1544	9.7509	24.6982	24.7016
2.449	9.0	114777	19.7143	2.2606	31.3824	9.9473	24.9422	24.9453

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0