Update README.md
Browse files
README.md
CHANGED
@@ -9,10 +9,11 @@ language:
|
|
9 |
license: mit
|
10 |
metrics:
|
11 |
- sacrebleu
|
12 |
-
-
|
13 |
-
-
|
14 |
-
-
|
15 |
-
-
|
|
|
16 |
- "Automated Readability Index"
|
17 |
tags:
|
18 |
- "text2text generation"
|
@@ -39,7 +40,7 @@ widget:
|
|
39 |
Scientific Abstract Simplification-baseline *translates* hard-to-read scientific abstracts😵 into more accessible language😇. We hope it can make scientific knowledge accessible for everyone🤗.
|
40 |
|
41 |
Try it now with the Hosted inference API on the right.
|
42 |
-
You can choose an existing example or paste in any (perhaps full-of-jargon) abstract. Remember to prepend the instruction
|
43 |
|
44 |
|
45 |
# Model Details
|
@@ -63,9 +64,9 @@ As an ongoing effort, we are working on re-contextualizating abstracts for bette
|
|
63 |
|
64 |
- **Model type:** Language model
|
65 |
- **Developed by:**
|
66 |
-
-
|
67 |
- Fellows: Haining Wang and Deanna Zarrillo
|
68 |
-
- [LEADING](https://cci.drexel.edu/mrc/leading/) Montana State University Library
|
69 |
- **Language(s) (NLP):** English
|
70 |
- **License:** MIT
|
71 |
- **Parent Model:** [FLAN-T5-large](https://huggingface.co/google/flan-t5-large)
|
@@ -74,7 +75,7 @@ As an ongoing effort, we are working on re-contextualizating abstracts for bette
|
|
74 |
|
75 |
# Usage
|
76 |
|
77 |
-
Use the code below to get started with the model.
|
78 |
|
79 |
```python
|
80 |
import torch
|
@@ -94,7 +95,7 @@ encoding = tokenizer(INSTRUCTION + input_text,
|
|
94 |
return_tensors='pt')
|
95 |
decoded_ids = model.generate(input_ids=encoding['input_ids'],
|
96 |
attention_mask=encoding['attention_mask'],
|
97 |
-
|
98 |
top_p=.9,
|
99 |
do_sample=True)
|
100 |
|
@@ -134,14 +135,14 @@ The model is evaluated on the SAS test set using the following metrics.
|
|
134 |
- [The Automated Readability Index (ARI)](https://www.readabilityformulas.com/automated-readability-index.php): ARI is a readability test designed to assess the understandability of a text. Like other popular readability formulas, the ARI formula outputs a number which approximates the grade level needed to comprehend the text. For example, if the ARI outputs the number 10, this equates to a high school student, ages 15-16 years old; a number 3 means students in 3rd grade (ages 8-9 yrs. old) should be able to comprehend the text.
|
135 |
|
136 |
|
137 |
-
Implementations of
|
138 |
|
139 |
|
140 |
## Results
|
141 |
|
142 |
| Metrics | SAS-baseline |
|
143 |
|----------------|-------------------|
|
144 |
-
|
|
145 |
| BERT Score F1↑ | 0.89 |
|
146 |
| ROUGLE-1↑ | 0.48 |
|
147 |
| ROUGLE-2↑ | 0.23 |
|
|
|
9 |
license: mit
|
10 |
metrics:
|
11 |
- sacrebleu
|
12 |
+
- bert_score
|
13 |
+
- rouge
|
14 |
+
- meteor
|
15 |
+
- sari
|
16 |
+
- ari
|
17 |
- "Automated Readability Index"
|
18 |
tags:
|
19 |
- "text2text generation"
|
|
|
40 |
Scientific Abstract Simplification-baseline *translates* hard-to-read scientific abstracts😵 into more accessible language😇. We hope it can make scientific knowledge accessible for everyone🤗.
|
41 |
|
42 |
Try it now with the Hosted inference API on the right.
|
43 |
+
You can choose an existing example or paste in any (perhaps full-of-jargon) abstract. Remember to prepend the instruction to the abstract ("summarize, simplify, and contextualize: "; notice, there is a whitespace after the colon). Local use refers to Section [Usage](/Usage).
|
44 |
|
45 |
|
46 |
# Model Details
|
|
|
64 |
|
65 |
- **Model type:** Language model
|
66 |
- **Developed by:**
|
67 |
+
- PIs: Jason Clark and Hannah McKelvey
|
68 |
- Fellows: Haining Wang and Deanna Zarrillo
|
69 |
+
- [LEADING](https://cci.drexel.edu/mrc/leading/) Montana State University Library, Project "TL;DR it": Automating Article Synopses for Search Engine Optimization and Citizen Science
|
70 |
- **Language(s) (NLP):** English
|
71 |
- **License:** MIT
|
72 |
- **Parent Model:** [FLAN-T5-large](https://huggingface.co/google/flan-t5-large)
|
|
|
75 |
|
76 |
# Usage
|
77 |
|
78 |
+
Use the code below to get started with the model. Remember to prepend the `INSTRUCTION` for best performance.
|
79 |
|
80 |
```python
|
81 |
import torch
|
|
|
95 |
return_tensors='pt')
|
96 |
decoded_ids = model.generate(input_ids=encoding['input_ids'],
|
97 |
attention_mask=encoding['attention_mask'],
|
98 |
+
max_length=512,
|
99 |
top_p=.9,
|
100 |
do_sample=True)
|
101 |
|
|
|
135 |
- [The Automated Readability Index (ARI)](https://www.readabilityformulas.com/automated-readability-index.php): ARI is a readability test designed to assess the understandability of a text. Like other popular readability formulas, the ARI formula outputs a number which approximates the grade level needed to comprehend the text. For example, if the ARI outputs the number 10, this equates to a high school student, ages 15-16 years old; a number 3 means students in 3rd grade (ages 8-9 yrs. old) should be able to comprehend the text.
|
136 |
|
137 |
|
138 |
+
Implementations of SacreBLEU, BERT Score, ROUGLE, METEOR, and SARI are from Huggingface [`evaluate`](https://pypi.org/project/evaluate/) v.0.3.0. ARI is from [`py-readability-metrics`](https://pypi.org/project/py-readability-metrics/) v.1.4.5.
|
139 |
|
140 |
|
141 |
## Results
|
142 |
|
143 |
| Metrics | SAS-baseline |
|
144 |
|----------------|-------------------|
|
145 |
+
| SacreBLEU↑ | 20.97 |
|
146 |
| BERT Score F1↑ | 0.89 |
|
147 |
| ROUGLE-1↑ | 0.48 |
|
148 |
| ROUGLE-2↑ | 0.23 |
|