Hugging Face's logo

CodeIsAbstract
/
denoice-finetuned-xsum

Text2Text Generation
Transformers
Safetensors
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
denoice-finetuned-xsum / README.md
CodeIsAbstract's picture
CodeIsAbstract
Model save
08ef55a verified
preview code
|
raw
history blame
8.69 kB
metadata
license: apache-2.0
base_model: google/t5-efficient-tiny
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: denoice-finetuned-xsum
    results: []

denoice-finetuned-xsum

This model is a fine-tuned version of google/t5-efficient-tiny on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0289
  • Rouge1: 91.6854
  • Rouge2: 69.1011
  • Rougel: 91.7603
  • Rougelsum: 91.9288
  • Gen Len: 5.0843

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 500
  • eval_batch_size: 500
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 70

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 36 0.0428 89.4757 66.4794 89.1948 89.3633 5.1236
No log 2.0 72 0.0420 90.0187 67.603 89.7753 89.9157 5.1348
No log 3.0 108 0.0417 90.0187 67.603 89.7753 89.9157 5.1348
No log 4.0 144 0.0411 89.7472 67.0412 89.5318 89.6442 5.1348
No log 5.0 180 0.0405 90.0094 67.0412 89.8408 89.9157 5.118
No log 6.0 216 0.0400 90.1498 67.4157 90.0187 90.0936 5.1292
No log 7.0 252 0.0396 90.4775 67.9775 90.2809 90.3839 5.1292
No log 8.0 288 0.0392 90.4775 67.9775 90.2809 90.3839 5.1292
No log 9.0 324 0.0389 90.4775 67.9775 90.2809 90.3839 5.1292
No log 10.0 360 0.0384 90.4775 67.9775 90.2809 90.3839 5.1292
No log 11.0 396 0.0378 90.7491 68.4457 90.5337 90.6929 5.118
No log 12.0 432 0.0374 90.3464 68.4457 90.2809 90.4682 5.118
No log 13.0 468 0.0372 90.7491 68.4457 90.5337 90.6929 5.118
0.075 14.0 504 0.0365 90.8614 69.1011 90.8614 90.9925 5.1124
0.075 15.0 540 0.0363 90.8614 69.1011 90.8614 90.9925 5.1124
0.075 16.0 576 0.0359 90.8614 69.1011 90.8614 90.9925 5.1124
0.075 17.0 612 0.0355 90.8614 69.1011 90.8614 90.9925 5.1124
0.075 18.0 648 0.0354 90.8614 69.1011 90.8614 90.9925 5.1124
0.075 19.0 684 0.0352 90.8614 69.1011 90.8614 90.9925 5.1011
0.075 20.0 720 0.0350 90.8614 69.1011 90.8614 90.9925 5.1011
0.075 21.0 756 0.0347 90.8614 69.1011 90.8614 90.9925 5.1011
0.075 22.0 792 0.0343 90.7303 68.867 90.7678 90.8708 5.1067
0.075 23.0 828 0.0340 90.7303 68.867 90.7678 90.8708 5.1067
0.075 24.0 864 0.0335 90.7303 68.867 90.7678 90.8708 5.1067
0.075 25.0 900 0.0333 90.7303 68.867 90.7678 90.8708 5.1067
0.075 26.0 936 0.0332 90.7303 68.867 90.7678 90.8708 5.1067
0.075 27.0 972 0.0328 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 28.0 1008 0.0327 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 29.0 1044 0.0324 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 30.0 1080 0.0325 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 31.0 1116 0.0322 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 32.0 1152 0.0321 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 33.0 1188 0.0319 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 34.0 1224 0.0317 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 35.0 1260 0.0316 90.7303 68.867 90.7678 90.8708 5.1067
0.0645 36.0 1296 0.0315 90.7303 68.867 90.7678 90.8708 5.0955
0.0645 37.0 1332 0.0313 90.7303 68.867 90.7678 90.8708 5.0955
0.0645 38.0 1368 0.0312 90.7303 68.867 90.7678 90.8708 5.0955
0.0645 39.0 1404 0.0311 90.7303 68.867 90.7678 90.8708 5.0955
0.0645 40.0 1440 0.0309 90.7303 68.867 90.7678 90.8708 5.0955
0.0645 41.0 1476 0.0307 90.7303 68.867 90.7678 90.8708 5.0955
0.0583 42.0 1512 0.0305 90.7303 68.867 90.7678 90.8708 5.0955
0.0583 43.0 1548 0.0304 91.2547 68.867 91.236 91.4794 5.0955
0.0583 44.0 1584 0.0305 91.2547 68.867 91.236 91.4794 5.0955
0.0583 45.0 1620 0.0304 91.4888 68.867 91.5356 91.7603 5.0843
0.0583 46.0 1656 0.0302 91.4888 68.867 91.5356 91.7603 5.0843
0.0583 47.0 1692 0.0300 91.4888 68.867 91.5356 91.7603 5.0843
0.0583 48.0 1728 0.0300 91.6854 69.1011 91.7603 91.9288 5.0843
0.0583 49.0 1764 0.0298 91.6854 69.1011 91.7603 91.9288 5.0843
0.0583 50.0 1800 0.0297 91.6854 69.1011 91.7603 91.9288 5.0843
0.0583 51.0 1836 0.0296 91.6854 69.1011 91.7603 91.9288 5.0843
0.0583 52.0 1872 0.0296 91.6854 69.1011 91.7603 91.9288 5.0843
0.0583 53.0 1908 0.0295 91.6854 69.1011 91.7603 91.9288 5.0843
0.0583 54.0 1944 0.0295 91.6854 69.1011 91.7603 91.9288 5.0843
0.0583 55.0 1980 0.0294 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 56.0 2016 0.0294 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 57.0 2052 0.0293 91.6854 69.1011 91.7603 91.9288 5.0787
0.0548 58.0 2088 0.0292 91.6854 69.1011 91.7603 91.9288 5.0787
0.0548 59.0 2124 0.0292 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 60.0 2160 0.0291 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 61.0 2196 0.0291 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 62.0 2232 0.0291 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 63.0 2268 0.0290 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 64.0 2304 0.0290 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 65.0 2340 0.0290 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 66.0 2376 0.0289 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 67.0 2412 0.0289 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 68.0 2448 0.0289 91.6854 69.1011 91.7603 91.9288 5.0843
0.0548 69.0 2484 0.0289 91.6854 69.1011 91.7603 91.9288 5.0843
0.0527 70.0 2520 0.0289 91.6854 69.1011 91.7603 91.9288 5.0843

Framework versions

  • Transformers 4.36.2
  • Pytorch 1.13.1
  • Datasets 2.16.1
  • Tokenizers 0.15.0