Skolkovo Institute of Science and Technology
commited on
Commit
·
d99ec6e
1
Parent(s):
fc34a90
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
+
|
5 |
+
tags:
|
6 |
+
- formality transfer
|
7 |
+
|
8 |
+
licenses:
|
9 |
+
- cc-by-nc-sa
|
10 |
+
---
|
11 |
+
LEWIP-informal model is designed to transfer formal text into informal keeping the important slots from the source text. The slots can be either pre-defined or detected automatically.
|
12 |
+
|
13 |
+
The model is based on LEWIP (Levenshtein editing with the parallel corpus). It exploits the ability of T5 model to fill the slots between the known texts with undefined number of tokens.
|
14 |
+
|
15 |
+
## How to use
|
16 |
+
|
17 |
+
### Installation
|
18 |
+
|
19 |
+
```python
|
20 |
+
!pip install lewip_informal
|
21 |
+
```
|
22 |
+
|
23 |
+
### Generating content preserving informal paraphrases
|
24 |
+
|
25 |
+
#### When the important entities are known in advance
|
26 |
+
|
27 |
+
```python
|
28 |
+
from lewip_informal import LEWIP
|
29 |
+
model = LEWIP(use_cuda = True)
|
30 |
+
text = 'I want to go to NY'
|
31 |
+
ent = ['NY']
|
32 |
+
model.generate(text, ent)
|
33 |
+
# expected output 'i wanna go to NY'
|
34 |
+
```
|
35 |
+
|
36 |
+
#### When the important entities are NOT known in advance
|
37 |
+
|
38 |
+
In case the important slots are not known, they are automatically detected with auxiliary tagger model.
|
39 |
+
|
40 |
+
```python
|
41 |
+
from lewip_informal import LEWIP
|
42 |
+
model = LEWIP(predefined_entities = False, use_cuda = True)
|
43 |
+
text = 'I really want to travel to NY'
|
44 |
+
model.generate(text)
|
45 |
+
# expected output 'I really want to go to NY'
|
46 |
+
```
|
47 |
+
|
48 |
+
You may want to have a look at the slots which were labeled as important by the tagger. Use 'show_template' variable
|
49 |
+
|
50 |
+
```python
|
51 |
+
|
52 |
+
model.generate(text, show_template = True)
|
53 |
+
# expected output
|
54 |
+
# I <extra_id_0> want to <extra_id_1> to NY
|
55 |
+
# 'I really want to go to NY'
|
56 |
+
|
57 |
+
```
|