Hugging Face's logo

CodeIsAbstract
/
denoice-finetuned-xsum

Text2Text Generation
Transformers
Safetensors
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
denoice-finetuned-xsum / README.md
CodeIsAbstract's picture
CodeIsAbstract
Model save
a4b7169 verified
preview code
|
raw
history blame
8.69 kB
metadata
license: apache-2.0
base_model: google/t5-efficient-tiny
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: denoice-finetuned-xsum
    results: []

denoice-finetuned-xsum

This model is a fine-tuned version of google/t5-efficient-tiny on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0221
  • Rouge1: 94.8315
  • Rouge2: 72.6592
  • Rougel: 94.8315
  • Rougelsum: 94.8876
  • Gen Len: 5.1348

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 500
  • eval_batch_size: 500
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 70

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 36 0.0285 91.8165 69.4757 91.8727 92.0412 5.0843
No log 2.0 72 0.0284 91.6854 69.2884 91.7603 91.9476 5.0787
No log 3.0 108 0.0280 92.3783 70.0375 92.4625 92.5281 5.0899
No log 4.0 144 0.0277 92.3783 70.0375 92.4625 92.5281 5.0899
No log 5.0 180 0.0276 92.9307 70.412 93.0337 93.1086 5.0955
No log 6.0 216 0.0275 93.8764 71.5356 93.9607 94.0449 5.1236
No log 7.0 252 0.0271 92.9963 70.412 92.9588 93.0337 5.1067
No log 8.0 288 0.0268 93.3708 70.9738 93.427 93.3708 5.118
No log 9.0 324 0.0267 93.3708 70.9738 93.427 93.3708 5.118
No log 10.0 360 0.0264 93.3708 70.9738 93.427 93.3708 5.118
No log 11.0 396 0.0264 93.5581 70.9738 93.5393 93.6517 5.1124
No log 12.0 432 0.0262 93.5581 70.9738 93.5393 93.6517 5.1124
No log 13.0 468 0.0260 93.5581 70.9738 93.5393 93.6517 5.1124
0.049 14.0 504 0.0259 93.4644 70.8801 93.4457 93.5768 5.1236
0.049 15.0 540 0.0257 93.5581 70.9738 93.5393 93.6517 5.1124
0.049 16.0 576 0.0256 93.4644 70.8801 93.4457 93.5768 5.1236
0.049 17.0 612 0.0256 93.5581 70.9738 93.5393 93.6517 5.1124
0.049 18.0 648 0.0255 93.4644 70.8801 93.4457 93.5768 5.118
0.049 19.0 684 0.0252 93.5581 70.9738 93.5393 93.6517 5.1067
0.049 20.0 720 0.0250 93.5581 70.9738 93.5393 93.6517 5.1124
0.049 21.0 756 0.0248 93.5581 70.9738 93.5393 93.6517 5.1124
0.049 22.0 792 0.0245 93.8764 71.5356 93.9607 94.0449 5.118
0.049 23.0 828 0.0246 93.8764 71.5356 93.9607 94.0449 5.118
0.049 24.0 864 0.0246 93.5581 70.9738 93.5393 93.6517 5.1067
0.049 25.0 900 0.0245 93.8764 71.5356 93.9607 94.0449 5.118
0.049 26.0 936 0.0243 94.1573 72.2846 94.2697 94.2697 5.1236
0.049 27.0 972 0.0243 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 28.0 1008 0.0242 93.8764 71.5356 93.9607 94.0449 5.1236
0.0433 29.0 1044 0.0239 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 30.0 1080 0.0237 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 31.0 1116 0.0236 93.8764 71.5356 93.9607 94.0449 5.1236
0.0433 32.0 1152 0.0235 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 33.0 1188 0.0234 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 34.0 1224 0.0234 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 35.0 1260 0.0232 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 36.0 1296 0.0232 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 37.0 1332 0.0232 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 38.0 1368 0.0232 93.8764 71.5356 93.9607 94.0449 5.118
0.0433 39.0 1404 0.0230 93.8764 71.5356 93.9607 94.0449 5.1236
0.0433 40.0 1440 0.0228 93.8764 71.5356 93.9607 94.0449 5.1236
0.0433 41.0 1476 0.0228 94.4944 72.0974 94.4944 94.6067 5.1348
0.0399 42.0 1512 0.0228 94.4944 72.0974 94.4944 94.6067 5.1348
0.0399 43.0 1548 0.0228 94.4944 72.0974 94.4944 94.6067 5.1348
0.0399 44.0 1584 0.0228 94.4944 72.0974 94.4944 94.6067 5.1348
0.0399 45.0 1620 0.0228 94.4944 72.0974 94.4944 94.6067 5.1292
0.0399 46.0 1656 0.0228 94.4944 72.0974 94.4944 94.6067 5.1292
0.0399 47.0 1692 0.0228 94.4944 72.0974 94.4944 94.6067 5.1292
0.0399 48.0 1728 0.0227 94.8315 72.6592 94.8315 94.8876 5.1348
0.0399 49.0 1764 0.0226 94.8315 72.6592 94.8315 94.8876 5.1348
0.0399 50.0 1800 0.0225 94.8315 72.6592 94.8315 94.8876 5.1348
0.0399 51.0 1836 0.0224 94.8315 72.6592 94.8315 94.8876 5.1348
0.0399 52.0 1872 0.0225 94.8315 72.6592 94.8315 94.8876 5.1348
0.0399 53.0 1908 0.0224 94.8315 72.6592 94.8315 94.8876 5.1348
0.0399 54.0 1944 0.0224 94.8315 72.6592 94.8315 94.8876 5.1348
0.0399 55.0 1980 0.0224 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 56.0 2016 0.0223 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 57.0 2052 0.0222 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 58.0 2088 0.0222 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 59.0 2124 0.0222 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 60.0 2160 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 61.0 2196 0.0221 94.4944 72.0974 94.4944 94.6067 5.1292
0.0379 62.0 2232 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 63.0 2268 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 64.0 2304 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 65.0 2340 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 66.0 2376 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 67.0 2412 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 68.0 2448 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0379 69.0 2484 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348
0.0369 70.0 2520 0.0221 94.8315 72.6592 94.8315 94.8876 5.1348

Framework versions

  • Transformers 4.36.2
  • Pytorch 1.13.1
  • Datasets 2.16.1
  • Tokenizers 0.15.0