File size: 7,767 Bytes
59f10fd 7dbb753 59f10fd e2409f1 7dbb753 e2409f1 3ae64f1 7dbb753 770f12d 52e1724 17ba185 6e6d259 59f10fd 7b0f55e cd0a6e9 7b0f55e e8db495 59f10fd b847bc2 59f10fd 7ff153f 59f10fd a17c48a 59f10fd 7ff153f 727fa43 7ff153f 59f10fd 7ff153f 59f10fd 10d46e0 dc0e629 e8db495 59f10fd 6b09903 419b891 6b09903 59f10fd e2409f1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 |
---
language:
- en
license: apache-2.0
tags:
- generated_from_keras_callback
datasets:
- Babelscape/multinerd
metrics:
- seqeval
base_model: distilbert-base-uncased
pipeline_tag: token-classification
widget:
- text: After months of meticulous review and analysis, I am proud to present a study
that explores the deep connections between Epstein-Barr virus (EBV), Long COVID
and Myalgic Encephalomyelitis.
example_title: Example 1
- text: The boy is, of course, Cupid. The image of a cupid riding a lion was a common
theme in classical and Renaissance art, representing the Virgilian maxim Amor
vincit omnia – love conquers all.
example_title: Example 2
- text: Billionaire Charlie Munger, Warren Buffet's right hand man, dies at 99.
example_title: Example 3
model-index:
- name: i-be-snek/distilbert-base-uncased-finetuned-ner-exp_A
results:
- task:
type: token-classification
name: ner
dataset:
name: Babelscape/multinerd
type: Babelscape/multinerd
split: test
metrics:
- type: seqeval
value: 0.9053582270795385
name: precision
- type: seqeval
value: 0.9303178007408852
name: recall
- type: seqeval
value: 0.9176683270188665
name: f1
- type: seqeval
value: 0.9863554498955407
name: accuracy
---
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->
# i-be-snek/distilbert-base-uncased-finetuned-ner-exp_A
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the English subset of all named entities in [Babelscape/multinerd](https://huggingface.co/datasets/Babelscape/multinerd) dataset.
It achieves the following results on the validation set:
- Train Loss: 0.0163
- Validation Loss: 0.1024
- Train Precision: 0.8763
- Train Recall: 0.8862
- Train F1: 0.8812
- Train Accuracy: 0.9750
- Epoch: 2
## Model description
[distilbert-base-uncased-finetuned-ner-exp_A](https://huggingface.co/i-be-snek/distilbert-base-uncased-finetuned-ner-exp_B) is a Named Entity Recognition model finetuned on [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased).
This model is uncased, so it makes no distinction between "sarah" and "Sarah".
## Training and evaluation data
This model has been evaluated on the English subset of the test set of [Babelscape/multinerd](https://huggingface.co/datasets/Babelscape/multinerd)
### Evaluation results
| metric | value |
|:----------|---------:|
| precision | 0.905358 |
| recall | 0.930318 |
| f1 | 0.917668 |
| accuracy | 0.986355 |
|metric/tag | ANIM | BIO | CEL | DIS | EVE | FOOD | INST | LOC | MEDIA | MYTH | ORG | PER | PLANT | TIME | VEHI |
|:----------|------------:|----------:|----------:|------------:|-----------:|------------:|----------:|-------------:|-----------:|----------:|------------:|-------------:|------------:|-----------:|----------:|
| precision | 0.667262 | 0.666667 | 0.508197 | 0.662324 | 0.896277 | 0.637809 | 0.642857 | 0.964137 | 0.931915 | 0.638889 | 0.941176 | 0.99033 | 0.558043 | 0.756579 | 0.735294 |
| recall | 0.698878 | 0.75 | 0.756098 | 0.803689 | 0.957386 | 0.637809 | 0.75 | 0.963656 | 0.956332 | 0.71875 | 0.962224 | 0.992023 | 0.752796 | 0.795848 | 0.78125 |
| f1 | 0.682704 | 0.705882 | 0.607843 | 0.72619 | 0.925824 | 0.637809 | 0.692308 | 0.963897 | 0.943966 | 0.676471 | 0.951584 | 0.991176 | 0.640952 | 0.775717 | 0.757576 |
| number | 3208 | 16 | 82 | 1518 | 704 | 1132 | 24 | 24048 | 916 | 64 | 6618 | 10530 | 1788 | 578 | 64 |
## Training procedure
All scripts for training can be found in this [GitHub repository](https://github.com/i-be-snek/rise-assignment-ner-finetune).
The model had early stopped watching its `val_loss`.
### Training hyperparameters
The following hyperparameters were used during training:
- optimizer:
```python
{
"name": "AdamWeightDecay",
"learning_rate": 2e-05,
"decay": 0.0,
"beta_1": 0.9,
"beta_2": 0.999,
"epsilon": 1e-07,
"amsgrad": False,
"weight_decay_rate": 0.0,
}
```
- training_precision: `float32`
### Training results
| Train Loss | Validation Loss | Train Precision | Train Recall | Train F1 | Train Accuracy | Epoch |
|:----------:|:---------------:|:---------------:|:------------:|:--------:|:--------------:|:-----:|
| 0.0709 | 0.0710 | 0.8563 | 0.8875 | 0.8716 | 0.9735 | 0 |
| 0.0295 | 0.0851 | 0.8743 | 0.8835 | 0.8789 | 0.9748 | 1 |
| 0.0163 | 0.1024 | 0.8763 | 0.8862 | 0.8812 | 0.9750 | 2 |
Epoch 0
| Named Entity | precision | recall | f1 |
|:----------:|:---------:|:---------:|:------:|
| ANIM | 0.699150 | 0.620124 | 0.657270 |
| BIO | 0.480000 | 0.782609 | 0.595041 |
| CEL | 0.815385 | 0.876033 | 0.844622 |
| DIS | 0.628939 | 0.806709 | 0.706818 |
| EVE | 0.898876 | 0.924855 | 0.911681 |
| FOOD | 0.624774 | 0.602266 | 0.613314 |
| INST | 0.467391 | 0.741379 | 0.573333 |
| LOC | 0.967354 | 0.969634 | 0.968493 |
| MEDIA | 0.911227 | 0.939856 | 0.925320 |
| MYTH | 0.941860 | 0.771429 | 0.848168 |
| ORG | 0.924471 | 0.937629 | 0.931003 |
| PER | 0.988699 | 0.990918 | 0.989807 |
| PLANT | 0.622521 | 0.781333 | 0.692944 |
| TIME | 0.743902 | 0.738499 | 0.741191 |
| VEHI | 0.785714 | 0.791367 | 0.788530 |
Epoch 1
| Named Entity | precision | recall | f1 |
|:----------:|:---------:|:---------:|:--------:|
| ANIM | 0.701040 | 0.747340 | 0.723450 |
| BIO | 0.422222 | 0.826087 | 0.558824 |
| CEL | 0.729167 | 0.867769 | 0.792453 |
| DIS | 0.731099 | 0.749794 | 0.740328 |
| EVE | 0.864865 | 0.924855 | 0.893855 |
| FOOD | 0.652865 | 0.572632 | 0.610122 |
| INST | 0.871795 | 0.586207 | 0.701031 |
| LOC | 0.968255 | 0.966143 | 0.967198 |
| MEDIA | 0.946346 | 0.918312 | 0.932118 |
| MYTH | 0.914894 | 0.819048 | 0.864322 |
| ORG | 0.906064 | 0.943582 | 0.924442 |
| PER | 0.990389 | 0.988367 | 0.989377 |
| PLANT | 0.625889 | 0.743556 | 0.679667 |
| TIME | 0.755981 | 0.765133 | 0.760529 |
| VEHI | 0.737500 | 0.848921 | 0.789298 |
Epoch 2
| Named Entity | precision | recall | f1 |
|:----------:|:---------:|:---------:|:--------:|
| ANIM | 0.730443 | 0.687057 | 0.708086 |
| BIO | 0.330882 | 0.978261 | 0.494505 |
| CEL | 0.798561 | 0.917355 | 0.853846 |
| DIS | 0.738108 | 0.750894 | 0.744446 |
| EVE | 0.904899 | 0.907514 | 0.906205 |
| FOOD | 0.628664 | 0.623184 | 0.625912 |
| INST | 0.533333 | 0.551724 | 0.542373 |
| LOC | 0.967915 | 0.973997 | 0.970946 |
| MEDIA | 0.949627 | 0.913824 | 0.931382 |
| MYTH | 0.910000 | 0.866667 | 0.887805 |
| ORG | 0.924920 | 0.934136 | 0.929505 |
| PER | 0.989506 | 0.991020 | 0.990263 |
| PLANT | 0.637648 | 0.742222 | 0.685972 |
| TIME | 0.766355 | 0.794189 | 0.780024 |
| VEHI | 0.818182 | 0.647482 | 0.722892 |
### Framework versions
- Transformers 4.35.2
- TensorFlow 2.14.0
- Datasets 2.15.0
- Tokenizers 0.15.0 |