sasingh192
commited on
Commit
•
eae4e05
1
Parent(s):
19b0570
Update README.md
Browse files
README.md
CHANGED
@@ -14,25 +14,43 @@ should probably proofread and complete it, then remove this comment. -->
|
|
14 |
|
15 |
# minilm-finetuned-movie
|
16 |
|
17 |
-
This model is a fine-tuned version of [microsoft/miniLM-L12-H384-uncased](https://huggingface.co/microsoft/miniLM-L12-H384-uncased) on
|
18 |
It achieves the following results on the evaluation set:
|
19 |
- Loss: 0.0451
|
20 |
- F1: 0.9856
|
21 |
|
22 |
## Model description
|
23 |
|
24 |
-
|
|
|
|
|
|
|
|
|
|
|
25 |
|
26 |
## Intended uses & limitations
|
27 |
|
28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
29 |
|
30 |
## Training and evaluation data
|
31 |
|
32 |
-
|
|
|
33 |
|
34 |
## Training procedure
|
35 |
|
|
|
|
|
36 |
### Training hyperparameters
|
37 |
|
38 |
The following hyperparameters were used during training:
|
|
|
14 |
|
15 |
# minilm-finetuned-movie
|
16 |
|
17 |
+
This model is a fine-tuned version of [microsoft/miniLM-L12-H384-uncased](https://huggingface.co/microsoft/miniLM-L12-H384-uncased) on sasingh192/movie-review dataset.
|
18 |
It achieves the following results on the evaluation set:
|
19 |
- Loss: 0.0451
|
20 |
- F1: 0.9856
|
21 |
|
22 |
## Model description
|
23 |
|
24 |
+
This model can be used to categorize a movie review into of the following categories:
|
25 |
+
0 - negative
|
26 |
+
1 - somewhat negative
|
27 |
+
2 - neutral
|
28 |
+
3 - somewhat positive
|
29 |
+
4 - positive
|
30 |
|
31 |
## Intended uses & limitations
|
32 |
|
33 |
+
The fined model is based on the finetuning of the model devloped by Wang et al.
|
34 |
+
|
35 |
+
@misc{wang2020minilm,
|
36 |
+
title={MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers},
|
37 |
+
author={Wenhui Wang and Furu Wei and Li Dong and Hangbo Bao and Nan Yang and Ming Zhou},
|
38 |
+
year={2020},
|
39 |
+
eprint={2002.10957},
|
40 |
+
archivePrefix={arXiv},
|
41 |
+
primaryClass={cs.CL}
|
42 |
+
}
|
43 |
+
|
44 |
|
45 |
## Training and evaluation data
|
46 |
|
47 |
+
sasingh192/movie-review dataset contains a column 'TrainValTest'. The values provied in this columns are 'Train', 'Val', and 'Test'.
|
48 |
+
The dataset can be filtered for the 'Train' values to train the model. Evaluation can be perfored on the data filtered by 'Val'. 'Test' is used as a blind test for kaggle.
|
49 |
|
50 |
## Training procedure
|
51 |
|
52 |
+
Training details are listed below.
|
53 |
+
|
54 |
### Training hyperparameters
|
55 |
|
56 |
The following hyperparameters were used during training:
|