metadata
license: mit
base_model: facebook/bart-large-xsum
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: text_shortening_model_v44
results: []
text_shortening_model_v44
This model is a fine-tuned version of facebook/bart-large-xsum on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.8836
- Rouge1: 0.4921
- Rouge2: 0.2719
- Rougel: 0.4429
- Rougelsum: 0.4423
- Bert precision: 0.8746
- Bert recall: 0.8761
- Average word count: 8.7063
- Max word count: 17
- Min word count: 5
- Average token count: 16.2989
- % shortened texts with length > 12: 8.7302
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Bert precision | Bert recall | Average word count | Max word count | Min word count | Average token count | % shortened texts with length > 12 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1.0083 | 1.0 | 83 | 1.4717 | 0.4904 | 0.2378 | 0.426 | 0.4266 | 0.8725 | 0.8732 | 8.5794 | 18 | 4 | 15.6164 | 6.3492 |
| 0.5702 | 2.0 | 166 | 1.4852 | 0.4722 | 0.2421 | 0.414 | 0.4143 | 0.869 | 0.8653 | 7.9101 | 14 | 4 | 13.6455 | 1.5873 |
| 0.4588 | 3.0 | 249 | 1.6283 | 0.5038 | 0.2733 | 0.4424 | 0.4422 | 0.8732 | 0.8794 | 9.0053 | 16 | 4 | 16.8386 | 8.9947 |
| 0.3586 | 4.0 | 332 | 1.6017 | 0.4965 | 0.2762 | 0.4381 | 0.4383 | 0.8709 | 0.8787 | 9.2381 | 18 | 4 | 16.3042 | 12.1693 |
| 0.2479 | 5.0 | 415 | 1.7497 | 0.4794 | 0.2613 | 0.4295 | 0.43 | 0.872 | 0.8702 | 8.3228 | 15 | 4 | 15.209 | 3.1746 |
| 0.2296 | 6.0 | 498 | 1.8482 | 0.4935 | 0.2739 | 0.4442 | 0.4443 | 0.8737 | 0.8755 | 8.7963 | 17 | 5 | 16.2989 | 7.1429 |
| 0.3065 | 7.0 | 581 | 1.9485 | 0.4765 | 0.2552 | 0.4213 | 0.4212 | 0.8698 | 0.8693 | 8.4683 | 17 | 5 | 15.6005 | 7.9365 |
| 0.2598 | 8.0 | 664 | 2.1608 | 0.4871 | 0.2585 | 0.4316 | 0.4319 | 0.8707 | 0.8736 | 8.963 | 16 | 5 | 16.6481 | 9.5238 |
| 0.2707 | 9.0 | 747 | 2.0966 | 0.4758 | 0.2603 | 0.4231 | 0.4246 | 0.8709 | 0.8717 | 8.4841 | 16 | 4 | 15.9312 | 7.1429 |
| 0.2099 | 10.0 | 830 | 2.2721 | 0.4777 | 0.2604 | 0.4246 | 0.4246 | 0.8735 | 0.8724 | 8.4312 | 15 | 4 | 15.9471 | 5.5556 |
| 0.1668 | 11.0 | 913 | 2.3536 | 0.4758 | 0.2541 | 0.4331 | 0.4328 | 0.8721 | 0.87 | 8.2857 | 14 | 4 | 15.7725 | 3.1746 |
| 0.1552 | 12.0 | 996 | 2.4572 | 0.484 | 0.2562 | 0.4313 | 0.4304 | 0.8726 | 0.875 | 8.828 | 17 | 4 | 16.246 | 7.9365 |
| 0.2141 | 13.0 | 1079 | 2.4485 | 0.4785 | 0.2631 | 0.4257 | 0.4252 | 0.8678 | 0.8736 | 9.1402 | 19 | 4 | 16.6561 | 11.3757 |
| 0.1348 | 14.0 | 1162 | 2.5012 | 0.4821 | 0.2613 | 0.4292 | 0.4296 | 0.8706 | 0.8738 | 8.8783 | 17 | 4 | 16.5185 | 10.0529 |
| 0.074 | 15.0 | 1245 | 2.5309 | 0.4915 | 0.2745 | 0.445 | 0.444 | 0.8764 | 0.8768 | 8.6667 | 16 | 4 | 16.2513 | 9.2593 |
| 0.1822 | 16.0 | 1328 | 2.5735 | 0.4709 | 0.2566 | 0.4239 | 0.4232 | 0.872 | 0.8692 | 8.2063 | 15 | 3 | 15.7249 | 4.2328 |
| 0.086 | 17.0 | 1411 | 2.8597 | 0.4831 | 0.2675 | 0.4373 | 0.4372 | 0.8722 | 0.8743 | 8.754 | 16 | 5 | 16.5476 | 8.7302 |
| 0.0872 | 18.0 | 1494 | 2.7420 | 0.4831 | 0.2677 | 0.4367 | 0.4353 | 0.8724 | 0.873 | 8.664 | 17 | 5 | 16.3016 | 7.672 |
| 0.1164 | 19.0 | 1577 | 2.8790 | 0.4867 | 0.269 | 0.4388 | 0.4381 | 0.8737 | 0.8755 | 8.7725 | 17 | 5 | 16.4418 | 8.9947 |
| 0.1101 | 20.0 | 1660 | 2.8836 | 0.4921 | 0.2719 | 0.4429 | 0.4423 | 0.8746 | 0.8761 | 8.7063 | 17 | 5 | 16.2989 | 8.7302 |
Framework versions
- Transformers 4.33.1
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3