Update README.md
Browse files
README.md
CHANGED
@@ -23,17 +23,26 @@ should probably proofread and complete it, then remove this comment. -->
|
|
23 |
|
24 |
This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the arynkiewicz/anydef-kilt-tasks-v2 dataset.
|
25 |
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
|
38 |
## Training procedure
|
39 |
|
|
|
23 |
|
24 |
This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the arynkiewicz/anydef-kilt-tasks-v2 dataset.
|
25 |
|
26 |
+
Find out about Model description, Intended uses & limitations and Training and evaluation data on our [github](https://github.com/daisd-ai/universal-el).
|
27 |
+
|
28 |
+
This is an updated version of the anydef model. The primary goal was to use an improved dataset during fine-tuning, enabling the model to better understand nuances.
|
29 |
+
Overall, anydef-v2 offers better performance in benchmarks, and manual inspection of the results suggests that the model has indeed improved.
|
30 |
+
|
31 |
+
Precision (%):
|
32 |
+
| Dataset | anydef | anydef-v2 |
|
33 |
+
|------------|------------|------------|
|
34 |
+
| RSS-500 | 66.23| 66.89|
|
35 |
+
| ISTEX-1000| 86.72| 85.82|
|
36 |
+
| Reuters-128| 63.8| 64.88|
|
37 |
+
| TweekiGold| 75.23| 75.93|
|
38 |
+
|
39 |
+
Retrieval rate (%):
|
40 |
+
| Dataset | anydef | anydef-v2 |
|
41 |
+
|------------|------------|------------|
|
42 |
+
| RSS-500 | 82.78| 84.11|
|
43 |
+
| ISTEX-1000| 97.91| 97.76|
|
44 |
+
| Reuters-128| 80.47| 83.33|
|
45 |
+
| TweekiGold| 89.93| 91.67|
|
46 |
|
47 |
## Training procedure
|
48 |
|