Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ This model was trained on ~25k heterogeneous manually annotated sentences (π
|
|
12 |
|
13 |
π **Dataset**
|
14 |
|
15 |
-
The dataset (π Stab et al. 2018) consists of **ARGUMENTS** (\~11k) that either support or oppose a topic if it includes a relevant reason for supporting or opposing the topic, or as a **NON-ARGUMENT** (\~14k) if it does not include reasons. The authors focus on controversial topics, i.e., topics that include an obvious polarity to the possible outcomes and compile a final set of eight controversial topics: _abortion, school uniforms, death penalty, marijuana legalization, nuclear energy, cloning, gun control, and minimum wage_.
|
16 |
|
17 |
| TOPIC | ARGUMENT | NON-ARGUMENT |
|
18 |
|----|----|----|
|
@@ -27,7 +27,7 @@ The dataset (π Stab et al. 2018) consists of **ARGUMENTS** (\~11k) that eithe
|
|
27 |
|
28 |
ππΌββοΈ**Model training**
|
29 |
|
30 |
-
**RoBERTArg** was fine-tuned on a RoBERTA (base) pre-trained model from HuggingFace using the HuggingFace trainer with the following hyperparameters
|
31 |
|
32 |
```
|
33 |
training_args = TrainingArguments(
|
@@ -41,13 +41,13 @@ training_args = TrainingArguments(
|
|
41 |
|
42 |
π **Evaluation**
|
43 |
|
44 |
-
The model was evaluated
|
45 |
|
46 |
| Model | Acc | F1 | R arg | R non | P arg | P non |
|
47 |
|----|----|----|----|----|----|----|
|
48 |
| RoBERTArg | 0.8193 | 0.8021 | 0.8463 | 0.7986 | 0.7623 | 0.8719 |
|
49 |
|
50 |
-
Showing the **confusion matrix** using
|
51 |
|
52 |
| | ARGUMENT | NON-ARGUMENT |
|
53 |
|----|----|----|
|
@@ -56,6 +56,6 @@ Showing the **confusion matrix** using the 20% of the sentences as an evaluation
|
|
56 |
|
57 |
β οΈ **Intended Uses & Potential Limitations**
|
58 |
|
59 |
-
The model can be a starting point to dive into the exciting
|
60 |
|
61 |
Enjoy and stay tuned! π
|
|
|
12 |
|
13 |
π **Dataset**
|
14 |
|
15 |
+
The dataset (π Stab et al. 2018) consists of **ARGUMENTS** (\~11k) that either support or oppose a topic if it includes a relevant reason for supporting or opposing the topic, or as a **NON-ARGUMENT** (\~14k) if it does not include reasons. The authors focus on controversial topics, i.e., topics that include "an obvious polarity to the possible outcomes" and compile a final set of eight controversial topics: _abortion, school uniforms, death penalty, marijuana legalization, nuclear energy, cloning, gun control, and minimum wage_.
|
16 |
|
17 |
| TOPIC | ARGUMENT | NON-ARGUMENT |
|
18 |
|----|----|----|
|
|
|
27 |
|
28 |
ππΌββοΈ**Model training**
|
29 |
|
30 |
+
**RoBERTArg** was fine-tuned on a RoBERTA (base) pre-trained model from HuggingFace using the HuggingFace trainer with the following hyperparameters:
|
31 |
|
32 |
```
|
33 |
training_args = TrainingArguments(
|
|
|
41 |
|
42 |
π **Evaluation**
|
43 |
|
44 |
+
The model was evaluated on an evaluation set (20%):
|
45 |
|
46 |
| Model | Acc | F1 | R arg | R non | P arg | P non |
|
47 |
|----|----|----|----|----|----|----|
|
48 |
| RoBERTArg | 0.8193 | 0.8021 | 0.8463 | 0.7986 | 0.7623 | 0.8719 |
|
49 |
|
50 |
+
Showing the **confusion matrix** using again the evaluation set:
|
51 |
|
52 |
| | ARGUMENT | NON-ARGUMENT |
|
53 |
|----|----|----|
|
|
|
56 |
|
57 |
β οΈ **Intended Uses & Potential Limitations**
|
58 |
|
59 |
+
The model can only be a starting point to dive into the exciting field of argument mining. But be aware. An argument is a complex structure, with multiple dependencies. Therefore, the model may perform less well on different topics and text types not included in the training set.
|
60 |
|
61 |
Enjoy and stay tuned! π
|