File size: 4,889 Bytes
258c454 4d7a41b 45a2786 4d7a41b f52be04 f7a9b56 258c454 f2f525b 258c454 9a08632 04682f6 c790326 04682f6 dbcf4ab 04682f6 c790326 368c55a 9a08632 dbcf4ab c790326 258c454 368c55a 83145d7 1d9bf24 258c454 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 |
---
tags:
- summarization
- summary
- booksum
- long-document
- long-form
license: apache-2.0
datasets:
- kmfoda/booksum
metrics:
- rouge
inference: false
model-index:
- name: pszemraj/long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP
results:
- task:
type: summarization
name: Summarization
dataset:
name: kmfoda/booksum
type: kmfoda/booksum
config: kmfoda--booksum
split: test
metrics:
- name: ROUGE-1
type: rouge
value: 35.9969
verified: true
- name: ROUGE-2
type: rouge
value: 5.9272
verified: true
- name: ROUGE-L
type: rouge
value: 16.0136
verified: true
- name: ROUGE-LSUM
type: rouge
value: 32.941
verified: true
- name: loss
type: loss
value: 2.9339466094970703
verified: true
- name: gen_len
type: gen_len
value: 283.7198
verified: true
- task:
type: summarization
name: Summarization
dataset:
name: samsum
type: samsum
config: samsum
split: test
metrics:
- name: ROUGE-1
type: rouge
value: 26.2412
verified: true
- name: ROUGE-2
type: rouge
value: 5.9791
verified: true
- name: ROUGE-L
type: rouge
value: 18.7467
verified: true
- name: ROUGE-LSUM
type: rouge
value: 22.5566
verified: true
- name: loss
type: loss
value: 2.877626895904541
verified: true
- name: gen_len
type: gen_len
value: 47.6532
verified: true
- task:
type: summarization
name: Summarization
dataset:
name: xsum
type: xsum
config: default
split: test
metrics:
- name: ROUGE-1
type: rouge
value: 19.3209
verified: true
- name: ROUGE-2
type: rouge
value: 2.7978
verified: true
- name: ROUGE-L
type: rouge
value: 12.5816
verified: true
- name: ROUGE-LSUM
type: rouge
value: 15.0239
verified: true
- name: loss
type: loss
value: 4.483709335327148
verified: true
- name: gen_len
type: gen_len
value: 82.729
verified: true
- task:
type: summarization
name: Summarization
dataset:
name: billsum
type: billsum
config: default
split: test
metrics:
- name: ROUGE-1
type: rouge
value: 36.5688
verified: true
- name: ROUGE-2
type: rouge
value: 12.5849
verified: true
- name: ROUGE-L
type: rouge
value: 22.2461
verified: true
- name: ROUGE-LSUM
type: rouge
value: 30.6507
verified: true
- name: loss
type: loss
value: 2.6456267833709717
verified: true
- name: gen_len
type: gen_len
value: 139.0398
verified: true
- task:
type: summarization
name: Summarization
dataset:
name: launch/gov_report
type: launch/gov_report
config: plain_text
split: test
metrics:
- name: ROUGE-1
type: rouge
value: 37.0248
verified: true
- name: ROUGE-2
type: rouge
value: 9.0446
verified: true
- name: ROUGE-L
type: rouge
value: 18.0521
verified: true
- name: ROUGE-LSUM
type: rouge
value: 33.4723
verified: true
- name: loss
type: loss
value: 3.381495237350464
verified: true
- name: gen_len
type: gen_len
value: 211.2066
verified: true
---
# long-t5-tglobal-large-pubmed-3k-booksum-16384-WIP
> NOTE: this is still a work-in-progress (WIP) and not completed/converged by any means, but sharing to maybe save some time for others :)
## Updates
_As I update this WIP checkpoint, I will post a note here._
- July 26, 2022: add two more epochs of training, metrics starting to be _almost_ as good as the more-tuned `base` variant
- July 8, 2022: add checkpoint with ~4 epochs of training on A100, equating to approx 350 steps of functional batch size 128
- July 4, 2022: add checkpoint with six additional epochs of training with the dataset summary outputs filtered to 1024 **tokens**, resolving the prior issue of short summaries.
## About
- a checkpoint of [Stancld/longt5-tglobal-large-16384-pubmed-3k_steps](https://huggingface.co/Stancld/longt5-tglobal-large-16384-pubmed-3k_steps) trained on `kmfoda/booksum` for about 26 epochs
- max input lengths during training vary between 8192 and 16384 tokens depending on GPU availability. This checkpoint was **trained with 16384 tokens as the max input length for the final 10+ epochs**
## Comparisons
- compare to [pszemraj/led-large-book-summary](https://huggingface.co/pszemraj/led-large-book-summary).
- **inference API has been disabled because it's too compute-intensive :/**
|