Hugging Face's logo

dl-ru
/
rubert-electra-srl

Token Classification
Transformers
PyTorch
Safetensors
electra
Generated from Trainer
Inference Endpoints
Model card Files Community
dl-ru commited on
Commit
cb7c402
•
1 Parent(s): 489f210

New version with explicit predicate marking

Browse files
Files changed (4) hide show
  1. README.md +62 -58
  2. config.json +44 -40
  3. pytorch_model.bin +2 -2
  4. training_args.bin +1 -1
README.md CHANGED
@@ -15,55 +15,59 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [ai-forever/ruElectra-medium](https://huggingface.co/ai-forever/ruElectra-medium) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.1544
19
- - Addressee Precision: 1.0
20
- - Addressee Recall: 1.0
21
- - Addressee F1: 1.0
22
- - Addressee Number: 4
23
- - Benefactive Precision: 0.0
24
- - Benefactive Recall: 0.0
25
- - Benefactive F1: 0.0
26
- - Benefactive Number: 1
27
- - Causator Precision: 0.9
28
- - Causator Recall: 0.9
29
- - Causator F1: 0.9
30
- - Causator Number: 10
31
- - Cause Precision: 0.5385
32
- - Cause Recall: 0.7
33
- - Cause F1: 0.6087
34
- - Cause Number: 10
35
- - Contrsubject Precision: 0.6667
36
- - Contrsubject Recall: 0.875
37
- - Contrsubject F1: 0.7568
38
- - Contrsubject Number: 16
39
- - Deliberative Precision: 0.6
40
- - Deliberative Recall: 0.75
41
- - Deliberative F1: 0.6667
42
- - Deliberative Number: 4
43
- - Destinative Precision: 0.0
44
- - Destinative Recall: 0.0
45
- - Destinative F1: 0.0
46
- - Destinative Number: 3
47
- - Directivefinal Precision: 0.0
48
- - Directivefinal Recall: 0.0
49
- - Directivefinal F1: 0.0
50
- - Directivefinal Number: 1
51
- - Experiencer Precision: 0.7475
52
- - Experiencer Recall: 0.8810
53
- - Experiencer F1: 0.8087
54
- - Experiencer Number: 84
55
- - Instrument Precision: 0.0
56
- - Instrument Recall: 0.0
57
- - Instrument F1: 0.0
58
- - Instrument Number: 2
59
- - Object Precision: 0.8023
60
- - Object Recall: 0.8313
61
- - Object F1: 0.8166
62
- - Object Number: 166
63
- - Overall Precision: 0.7685
64
- - Overall Recall: 0.8272
65
- - Overall F1: 0.7968
66
- - Overall Accuracy: 0.9631
 
 
 
 
67
 
68
  ## Model description
69
 
@@ -86,21 +90,21 @@ The following hyperparameters were used during training:
86
  - train_batch_size: 4
87
  - eval_batch_size: 1
88
  - seed: 605573
89
- - gradient_accumulation_steps: 8
90
- - total_train_batch_size: 32
91
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
92
  - lr_scheduler_type: linear
93
  - num_epochs: 5
94
 
95
  ### Training results
96
 
97
- | Training Loss | Epoch | Step | Validation Loss | Addressee Precision | Addressee Recall | Addressee F1 | Addressee Number | Benefactive Precision | Benefactive Recall | Benefactive F1 | Benefactive Number | Causator Precision | Causator Recall | Causator F1 | Causator Number | Cause Precision | Cause Recall | Cause F1 | Cause Number | Contrsubject Precision | Contrsubject Recall | Contrsubject F1 | Contrsubject Number | Deliberative Precision | Deliberative Recall | Deliberative F1 | Deliberative Number | Destinative Precision | Destinative Recall | Destinative F1 | Destinative Number | Directivefinal Precision | Directivefinal Recall | Directivefinal F1 | Directivefinal Number | Experiencer Precision | Experiencer Recall | Experiencer F1 | Experiencer Number | Instrument Precision | Instrument Recall | Instrument F1 | Instrument Number | Object Precision | Object Recall | Object F1 | Object Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
98
- |:-------------:|:-----:|:----:|:---------------:|:-------------------:|:----------------:|:------------:|:----------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------------:|:---------------------:|:-----------------:|:---------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:----------------:|:-------------:|:---------:|:-------------:|:-----------------:|:--------------:|:----------:|:----------------:|
99
- | 0.2429 | 1.0 | 181 | 0.2241 | 1.0 | 0.25 | 0.4 | 4 | 0.0 | 0.0 | 0.0 | 1 | 0.0 | 0.0 | 0.0 | 10 | 0.0 | 0.0 | 0.0 | 10 | 0.0 | 0.0 | 0.0 | 16 | 0.0 | 0.0 | 0.0 | 4 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7143 | 0.8333 | 0.7692 | 84 | 0.0 | 0.0 | 0.0 | 2 | 0.5862 | 0.8193 | 0.6834 | 166 | 0.6254 | 0.6877 | 0.6551 | 0.9422 |
100
- | 0.1391 | 2.0 | 362 | 0.1603 | 1.0 | 1.0 | 1.0 | 4 | 0.0 | 0.0 | 0.0 | 1 | 0.8182 | 0.9 | 0.8571 | 10 | 0.7 | 0.7 | 0.7 | 10 | 0.6471 | 0.6875 | 0.6667 | 16 | 0.0 | 0.0 | 0.0 | 4 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.8072 | 0.7976 | 0.8024 | 84 | 0.0 | 0.0 | 0.0 | 2 | 0.8037 | 0.7892 | 0.7964 | 166 | 0.7951 | 0.7608 | 0.7776 | 0.9608 |
101
- | 0.0883 | 3.0 | 543 | 0.1392 | 0.75 | 0.75 | 0.75 | 4 | 0.0 | 0.0 | 0.0 | 1 | 1.0 | 0.9 | 0.9474 | 10 | 0.4375 | 0.7 | 0.5385 | 10 | 0.8125 | 0.8125 | 0.8125 | 16 | 1.0 | 0.5 | 0.6667 | 4 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.8046 | 0.8333 | 0.8187 | 84 | 0.0 | 0.0 | 0.0 | 2 | 0.8118 | 0.8313 | 0.8214 | 166 | 0.7961 | 0.8040 | 0.8000 | 0.9645 |
102
- | 0.0565 | 4.0 | 724 | 0.1439 | 0.8 | 1.0 | 0.8889 | 4 | 0.0 | 0.0 | 0.0 | 1 | 0.9 | 0.9 | 0.9 | 10 | 0.6154 | 0.8 | 0.6957 | 10 | 0.875 | 0.875 | 0.875 | 16 | 0.5 | 0.5 | 0.5 | 4 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.8 | 0.8571 | 0.8276 | 84 | 0.0 | 0.0 | 0.0 | 2 | 0.7933 | 0.8554 | 0.8232 | 166 | 0.7918 | 0.8339 | 0.8123 | 0.9655 |
103
- | 0.0495 | 5.0 | 905 | 0.1544 | 1.0 | 1.0 | 1.0 | 4 | 0.0 | 0.0 | 0.0 | 1 | 0.9 | 0.9 | 0.9 | 10 | 0.5385 | 0.7 | 0.6087 | 10 | 0.6667 | 0.875 | 0.7568 | 16 | 0.6 | 0.75 | 0.6667 | 4 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1 | 0.7475 | 0.8810 | 0.8087 | 84 | 0.0 | 0.0 | 0.0 | 2 | 0.8023 | 0.8313 | 0.8166 | 166 | 0.7685 | 0.8272 | 0.7968 | 0.9631 |
104
 
105
 
106
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [ai-forever/ruElectra-medium](https://huggingface.co/ai-forever/ruElectra-medium) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.1367
19
+ - Addressee Precision: 0.8793
20
+ - Addressee Recall: 0.8947
21
+ - Addressee F1: 0.8870
22
+ - Addressee Number: 57
23
+ - Benefactive Precision: 0.6
24
+ - Benefactive Recall: 0.3
25
+ - Benefactive F1: 0.4
26
+ - Benefactive Number: 10
27
+ - Causator Precision: 0.9296
28
+ - Causator Recall: 0.8049
29
+ - Causator F1: 0.8627
30
+ - Causator Number: 82
31
+ - Cause Precision: 0.5618
32
+ - Cause Recall: 0.7353
33
+ - Cause F1: 0.6369
34
+ - Cause Number: 68
35
+ - Contrsubject Precision: 0.8409
36
+ - Contrsubject Recall: 0.925
37
+ - Contrsubject F1: 0.8810
38
+ - Contrsubject Number: 120
39
+ - Deliberative Precision: 0.9074
40
+ - Deliberative Recall: 0.9423
41
+ - Deliberative F1: 0.9245
42
+ - Deliberative Number: 52
43
+ - Destinative Precision: 0.9130
44
+ - Destinative Recall: 0.875
45
+ - Destinative F1: 0.8936
46
+ - Destinative Number: 24
47
+ - Directivefinal Precision: 0.6154
48
+ - Directivefinal Recall: 0.6667
49
+ - Directivefinal F1: 0.64
50
+ - Directivefinal Number: 12
51
+ - Experiencer Precision: 0.8525
52
+ - Experiencer Recall: 0.8660
53
+ - Experiencer F1: 0.8592
54
+ - Experiencer Number: 694
55
+ - Instrument Precision: 1.0
56
+ - Instrument Recall: 0.1111
57
+ - Instrument F1: 0.2000
58
+ - Instrument Number: 9
59
+ - Mediative Precision: 0.0
60
+ - Mediative Recall: 0.0
61
+ - Mediative F1: 0.0
62
+ - Mediative Number: 1
63
+ - Object Precision: 0.8735
64
+ - Object Recall: 0.8924
65
+ - Object F1: 0.8828
66
+ - Object Number: 1524
67
+ - Overall Precision: 0.8571
68
+ - Overall Recall: 0.8749
69
+ - Overall F1: 0.8659
70
+ - Overall Accuracy: 0.9711
71
 
72
  ## Model description
73
 
 
90
  - train_batch_size: 4
91
  - eval_batch_size: 1
92
  - seed: 605573
93
+ - gradient_accumulation_steps: 2
94
+ - total_train_batch_size: 8
95
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
96
  - lr_scheduler_type: linear
97
  - num_epochs: 5
98
 
99
  ### Training results
100
 
101
+ | Training Loss | Epoch | Step | Validation Loss | Addressee Precision | Addressee Recall | Addressee F1 | Addressee Number | Benefactive Precision | Benefactive Recall | Benefactive F1 | Benefactive Number | Causator Precision | Causator Recall | Causator F1 | Causator Number | Cause Precision | Cause Recall | Cause F1 | Cause Number | Contrsubject Precision | Contrsubject Recall | Contrsubject F1 | Contrsubject Number | Deliberative Precision | Deliberative Recall | Deliberative F1 | Deliberative Number | Destinative Precision | Destinative Recall | Destinative F1 | Destinative Number | Directivefinal Precision | Directivefinal Recall | Directivefinal F1 | Directivefinal Number | Experiencer Precision | Experiencer Recall | Experiencer F1 | Experiencer Number | Instrument Precision | Instrument Recall | Instrument F1 | Instrument Number | Mediative Precision | Mediative Recall | Mediative F1 | Mediative Number | Object Precision | Object Recall | Object F1 | Object Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
102
+ |:-------------:|:-----:|:----:|:---------------:|:-------------------:|:----------------:|:------------:|:----------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------------:|:---------------------:|:-----------------:|:---------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:-------------------:|:----------------:|:------------:|:----------------:|:----------------:|:-------------:|:---------:|:-------------:|:-----------------:|:--------------:|:----------:|:----------------:|
103
+ | 0.1821 | 1.0 | 724 | 0.1479 | 0.5761 | 0.9298 | 0.7114 | 57 | 0.0 | 0.0 | 0.0 | 10 | 0.6867 | 0.6951 | 0.6909 | 82 | 0.72 | 0.2647 | 0.3871 | 68 | 0.8171 | 0.5583 | 0.6634 | 120 | 0.5111 | 0.4423 | 0.4742 | 52 | 0.0 | 0.0 | 0.0 | 24 | 0.0 | 0.0 | 0.0 | 12 | 0.8496 | 0.8141 | 0.8315 | 694 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 1 | 0.8183 | 0.8688 | 0.8428 | 1524 | 0.8073 | 0.7942 | 0.8007 | 0.9619 |
104
+ | 0.0938 | 2.0 | 1448 | 0.1384 | 0.6714 | 0.8246 | 0.7402 | 57 | 0.0 | 0.0 | 0.0 | 10 | 0.8649 | 0.7805 | 0.8205 | 82 | 0.5067 | 0.5588 | 0.5315 | 68 | 0.7329 | 0.8917 | 0.8045 | 120 | 0.5465 | 0.9038 | 0.6812 | 52 | 0.0 | 0.0 | 0.0 | 24 | 0.5556 | 0.4167 | 0.4762 | 12 | 0.7835 | 0.9179 | 0.8454 | 694 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 1 | 0.8329 | 0.8832 | 0.8573 | 1524 | 0.7930 | 0.8636 | 0.8268 | 0.9635 |
105
+ | 0.0627 | 3.0 | 2172 | 0.1194 | 0.8125 | 0.9123 | 0.8595 | 57 | 0.25 | 0.2 | 0.2222 | 10 | 0.9178 | 0.8171 | 0.8645 | 82 | 0.5 | 0.6176 | 0.5526 | 68 | 0.7343 | 0.875 | 0.7985 | 120 | 0.8980 | 0.8462 | 0.8713 | 52 | 0.8421 | 0.6667 | 0.7442 | 24 | 0.7273 | 0.6667 | 0.6957 | 12 | 0.8815 | 0.8357 | 0.8580 | 694 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 1 | 0.8579 | 0.8871 | 0.8723 | 1524 | 0.8447 | 0.8549 | 0.8498 | 0.9687 |
106
+ | 0.0501 | 4.0 | 2896 | 0.1276 | 0.8772 | 0.8772 | 0.8772 | 57 | 0.6667 | 0.4 | 0.5 | 10 | 0.9242 | 0.7439 | 0.8243 | 82 | 0.5604 | 0.75 | 0.6415 | 68 | 0.8409 | 0.925 | 0.8810 | 120 | 0.9245 | 0.9423 | 0.9333 | 52 | 0.9130 | 0.875 | 0.8936 | 24 | 0.6154 | 0.6667 | 0.64 | 12 | 0.8693 | 0.8530 | 0.8611 | 694 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 1 | 0.8773 | 0.8865 | 0.8819 | 1524 | 0.8633 | 0.8662 | 0.8647 | 0.9713 |
107
+ | 0.0205 | 5.0 | 3620 | 0.1367 | 0.8793 | 0.8947 | 0.8870 | 57 | 0.6 | 0.3 | 0.4 | 10 | 0.9296 | 0.8049 | 0.8627 | 82 | 0.5618 | 0.7353 | 0.6369 | 68 | 0.8409 | 0.925 | 0.8810 | 120 | 0.9074 | 0.9423 | 0.9245 | 52 | 0.9130 | 0.875 | 0.8936 | 24 | 0.6154 | 0.6667 | 0.64 | 12 | 0.8525 | 0.8660 | 0.8592 | 694 | 1.0 | 0.1111 | 0.2000 | 9 | 0.0 | 0.0 | 0.0 | 1 | 0.8735 | 0.8924 | 0.8828 | 1524 | 0.8571 | 0.8749 | 0.8659 | 0.9711 |
108
 
109
 
110
  ### Framework versions
config.json CHANGED
@@ -12,50 +12,54 @@
12
  "hidden_size": 576,
13
  "id2label": {
14
  "0": "O",
15
- "1": "B-Object",
16
- "2": "B-Experiencer",
17
- "3": "B-Cause",
18
- "4": "B-Deliberative",
19
- "5": "B-Causator",
20
- "6": "B-ContrSubject",
21
- "7": "B-Benefactive",
22
- "8": "B-Addressee",
23
- "9": "I-Object",
24
- "10": "B-Destinative",
25
- "11": "I-ContrSubject",
26
- "12": "B-Instrument",
27
- "13": "I-Deliberative",
28
- "14": "B-DirectiveFinal",
29
- "15": "B-Mediative",
30
- "16": "I-DirectiveFinal",
31
- "17": "B-DirectiveInitial",
32
- "18": "I-DirectiveInitial",
33
- "19": "I-Experiencer",
34
- "20": "I-Cause"
 
 
35
  },
36
  "initializer_range": 0.02,
37
  "intermediate_size": 2304,
38
  "label2id": {
39
- "B-Addressee": 8,
40
- "B-Benefactive": 7,
41
- "B-Causator": 5,
42
- "B-Cause": 3,
43
- "B-ContrSubject": 6,
44
- "B-Deliberative": 4,
45
- "B-Destinative": 10,
46
- "B-DirectiveFinal": 14,
47
- "B-DirectiveInitial": 17,
48
- "B-Experiencer": 2,
49
- "B-Instrument": 12,
50
- "B-Mediative": 15,
51
- "B-Object": 1,
52
- "I-Cause": 20,
53
- "I-ContrSubject": 11,
54
- "I-Deliberative": 13,
55
- "I-DirectiveFinal": 16,
56
- "I-DirectiveInitial": 18,
57
- "I-Experiencer": 19,
58
- "I-Object": 9,
 
 
59
  "O": 0
60
  },
61
  "layer_norm_eps": 1e-12,
 
12
  "hidden_size": 576,
13
  "id2label": {
14
  "0": "O",
15
+ "1": "B-Predicate",
16
+ "2": "B-Object",
17
+ "3": "B-Experiencer",
18
+ "4": "B-Cause",
19
+ "5": "B-Deliberative",
20
+ "6": "B-Causator",
21
+ "7": "B-ContrSubject",
22
+ "8": "B-Benefactive",
23
+ "9": "B-Addressee",
24
+ "10": "I-Object",
25
+ "11": "B-Destinative",
26
+ "12": "I-ContrSubject",
27
+ "13": "B-Instrument",
28
+ "14": "I-Deliberative",
29
+ "15": "B-Limitative",
30
+ "16": "B-DirectiveFinal",
31
+ "17": "B-Mediative",
32
+ "18": "I-DirectiveFinal",
33
+ "19": "B-DirectiveInitial",
34
+ "20": "I-DirectiveInitial",
35
+ "21": "I-Experiencer",
36
+ "22": "I-Cause"
37
  },
38
  "initializer_range": 0.02,
39
  "intermediate_size": 2304,
40
  "label2id": {
41
+ "B-Addressee": 9,
42
+ "B-Benefactive": 8,
43
+ "B-Causator": 6,
44
+ "B-Cause": 4,
45
+ "B-ContrSubject": 7,
46
+ "B-Deliberative": 5,
47
+ "B-Destinative": 11,
48
+ "B-DirectiveFinal": 16,
49
+ "B-DirectiveInitial": 19,
50
+ "B-Experiencer": 3,
51
+ "B-Instrument": 13,
52
+ "B-Limitative": 15,
53
+ "B-Mediative": 17,
54
+ "B-Object": 2,
55
+ "B-Predicate": 1,
56
+ "I-Cause": 22,
57
+ "I-ContrSubject": 12,
58
+ "I-Deliberative": 14,
59
+ "I-DirectiveFinal": 18,
60
+ "I-DirectiveInitial": 20,
61
+ "I-Experiencer": 21,
62
+ "I-Object": 10,
63
  "O": 0
64
  },
65
  "layer_norm_eps": 1e-12,
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0c2def8bc6fe4241603e192ecfb7bd3478a3c3f63c0cf7d1235bbe7dfa757f5b
3
- size 340224041
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49608e38da88f2cc7a74e91f5dfeeda4ac34f55947329da8a6b5521ca0cde33d
3
+ size 340228649
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9f632f67809521dd8209dbdc9d50d71b3738522fe8ceef3252448f506f87f465
3
  size 4155
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfdd72cd5d712ce859220138bff2efdb41b9156c57fc69e5ea01c3ae5b094122
3
  size 4155