|
--- |
|
license: apache-2.0 |
|
base_model: distilbert-base-uncased |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- sembr2023 |
|
metrics: |
|
- precision |
|
- recall |
|
- f1 |
|
- accuracy |
|
model-index: |
|
- name: sembr2023-distilbert-base-uncased |
|
results: |
|
- task: |
|
name: Token Classification |
|
type: token-classification |
|
dataset: |
|
name: sembr2023 |
|
type: sembr2023 |
|
config: sembr2023 |
|
split: test |
|
args: sembr2023 |
|
metrics: |
|
- name: Precision |
|
type: precision |
|
value: 0.7588342440801458 |
|
- name: Recall |
|
type: recall |
|
value: 0.8253863426231673 |
|
- name: F1 |
|
type: f1 |
|
value: 0.790712387700873 |
|
- name: Accuracy |
|
type: accuracy |
|
value: 0.9613411398987951 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# sembr2023-distilbert-base-uncased |
|
|
|
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the sembr2023 dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.2562 |
|
- Precision: 0.7588 |
|
- Recall: 0.8254 |
|
- F1: 0.7907 |
|
- Iou: 0.6539 |
|
- Accuracy: 0.9613 |
|
- Balanced Accuracy: 0.9000 |
|
- Overall Accuracy: 0.9468 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0001 |
|
- train_batch_size: 64 |
|
- eval_batch_size: 128 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: cosine |
|
- training_steps: 1000 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Iou | Accuracy | Balanced Accuracy | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:------:|:--------:|:-----------------:|:----------------:| |
|
| 0.4048 | 0.07 | 10 | 0.4102 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | |
|
| 0.3834 | 0.14 | 20 | 0.3950 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | |
|
| 0.3817 | 0.21 | 30 | 0.3616 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | |
|
| 0.2982 | 0.28 | 40 | 0.3127 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | |
|
| 0.2505 | 0.35 | 50 | 0.2716 | 0.6064 | 0.7118 | 0.6549 | 0.4869 | 0.9336 | 0.8335 | 0.9149 | |
|
| 0.2176 | 0.42 | 60 | 0.2119 | 0.8427 | 0.5653 | 0.6767 | 0.5114 | 0.9522 | 0.7775 | 0.9411 | |
|
| 0.2018 | 0.49 | 70 | 0.2031 | 0.7785 | 0.7056 | 0.7402 | 0.5876 | 0.9562 | 0.8430 | 0.9391 | |
|
| 0.1609 | 0.56 | 80 | 0.1825 | 0.8199 | 0.6992 | 0.7548 | 0.6061 | 0.9598 | 0.8422 | 0.9437 | |
|
| 0.153 | 0.62 | 90 | 0.1820 | 0.7830 | 0.7362 | 0.7589 | 0.6115 | 0.9586 | 0.8582 | 0.9421 | |
|
| 0.1336 | 0.69 | 100 | 0.1760 | 0.7955 | 0.7323 | 0.7626 | 0.6163 | 0.9597 | 0.8570 | 0.9447 | |
|
| 0.1351 | 0.76 | 110 | 0.1691 | 0.8314 | 0.7243 | 0.7742 | 0.6316 | 0.9626 | 0.8550 | 0.9460 | |
|
| 0.1393 | 0.83 | 120 | 0.1742 | 0.8028 | 0.7506 | 0.7758 | 0.6338 | 0.9616 | 0.8664 | 0.9454 | |
|
| 0.1013 | 0.9 | 130 | 0.1657 | 0.7991 | 0.7608 | 0.7795 | 0.6387 | 0.9619 | 0.8711 | 0.9454 | |
|
| 0.0969 | 0.97 | 140 | 0.1681 | 0.7926 | 0.7702 | 0.7812 | 0.6410 | 0.9618 | 0.8753 | 0.9461 | |
|
| 0.1009 | 1.04 | 150 | 0.1624 | 0.8247 | 0.7475 | 0.7842 | 0.6450 | 0.9636 | 0.8660 | 0.9485 | |
|
| 0.086 | 1.11 | 160 | 0.1532 | 0.8062 | 0.7793 | 0.7925 | 0.6564 | 0.9639 | 0.8806 | 0.9496 | |
|
| 0.0732 | 1.18 | 170 | 0.1710 | 0.7633 | 0.8185 | 0.7899 | 0.6528 | 0.9615 | 0.8969 | 0.9465 | |
|
| 0.0804 | 1.25 | 180 | 0.1731 | 0.8098 | 0.7735 | 0.7912 | 0.6546 | 0.9639 | 0.8779 | 0.9472 | |
|
| 0.0865 | 1.32 | 190 | 0.1659 | 0.7905 | 0.7863 | 0.7884 | 0.6507 | 0.9627 | 0.8830 | 0.9486 | |
|
| 0.0867 | 1.39 | 200 | 0.1716 | 0.7704 | 0.8082 | 0.7888 | 0.6513 | 0.9617 | 0.8924 | 0.9452 | |
|
| 0.0769 | 1.46 | 210 | 0.1644 | 0.8122 | 0.7678 | 0.7894 | 0.6520 | 0.9637 | 0.8753 | 0.9486 | |
|
| 0.0639 | 1.53 | 220 | 0.1668 | 0.7927 | 0.7786 | 0.7856 | 0.6469 | 0.9624 | 0.8794 | 0.9474 | |
|
| 0.0642 | 1.6 | 230 | 0.1669 | 0.7960 | 0.7830 | 0.7895 | 0.6521 | 0.9630 | 0.8818 | 0.9482 | |
|
| 0.0598 | 1.67 | 240 | 0.1776 | 0.7959 | 0.7802 | 0.7880 | 0.6501 | 0.9628 | 0.8804 | 0.9464 | |
|
| 0.0679 | 1.74 | 250 | 0.1938 | 0.7436 | 0.8338 | 0.7861 | 0.6476 | 0.9599 | 0.9030 | 0.9423 | |
|
| 0.0578 | 1.81 | 260 | 0.1669 | 0.7965 | 0.7855 | 0.7910 | 0.6542 | 0.9633 | 0.8830 | 0.9494 | |
|
| 0.0587 | 1.88 | 270 | 0.1750 | 0.7982 | 0.7833 | 0.7907 | 0.6538 | 0.9633 | 0.8820 | 0.9468 | |
|
| 0.0624 | 1.94 | 280 | 0.1759 | 0.7684 | 0.8249 | 0.7956 | 0.6606 | 0.9625 | 0.9004 | 0.9482 | |
|
| 0.0494 | 2.01 | 290 | 0.1868 | 0.7606 | 0.8194 | 0.7889 | 0.6514 | 0.9612 | 0.8972 | 0.9447 | |
|
| 0.0418 | 2.08 | 300 | 0.1746 | 0.8089 | 0.7769 | 0.7926 | 0.6564 | 0.9640 | 0.8795 | 0.9501 | |
|
| 0.0406 | 2.15 | 310 | 0.1900 | 0.7778 | 0.8157 | 0.7963 | 0.6616 | 0.9631 | 0.8966 | 0.9477 | |
|
| 0.0417 | 2.22 | 320 | 0.2066 | 0.7754 | 0.8165 | 0.7954 | 0.6603 | 0.9628 | 0.8968 | 0.9467 | |
|
| 0.0438 | 2.29 | 330 | 0.2037 | 0.7623 | 0.8282 | 0.7939 | 0.6582 | 0.9619 | 0.9015 | 0.9461 | |
|
| 0.0454 | 2.36 | 340 | 0.1882 | 0.7671 | 0.8221 | 0.7936 | 0.6579 | 0.9622 | 0.8989 | 0.9487 | |
|
| 0.0372 | 2.43 | 350 | 0.1896 | 0.7937 | 0.8082 | 0.8009 | 0.6679 | 0.9644 | 0.8939 | 0.9489 | |
|
| 0.0301 | 2.5 | 360 | 0.1846 | 0.8058 | 0.7913 | 0.7985 | 0.6646 | 0.9647 | 0.8864 | 0.9503 | |
|
| 0.0334 | 2.57 | 370 | 0.1867 | 0.7723 | 0.8212 | 0.7960 | 0.6611 | 0.9628 | 0.8988 | 0.9498 | |
|
| 0.038 | 2.64 | 380 | 0.2099 | 0.7538 | 0.8276 | 0.7890 | 0.6516 | 0.9608 | 0.9007 | 0.9463 | |
|
| 0.0427 | 2.71 | 390 | 0.2050 | 0.7625 | 0.8272 | 0.7935 | 0.6577 | 0.9619 | 0.9011 | 0.9460 | |
|
| 0.0373 | 2.78 | 400 | 0.2002 | 0.7779 | 0.8086 | 0.7930 | 0.6569 | 0.9626 | 0.8931 | 0.9487 | |
|
| 0.0308 | 2.85 | 410 | 0.2002 | 0.7501 | 0.8260 | 0.7863 | 0.6478 | 0.9603 | 0.8997 | 0.9449 | |
|
| 0.035 | 2.92 | 420 | 0.2034 | 0.7656 | 0.8173 | 0.7906 | 0.6537 | 0.9617 | 0.8965 | 0.9470 | |
|
| 0.0265 | 2.99 | 430 | 0.1877 | 0.7766 | 0.8202 | 0.7978 | 0.6637 | 0.9632 | 0.8987 | 0.9487 | |
|
| 0.0358 | 3.06 | 440 | 0.2140 | 0.7555 | 0.8300 | 0.7910 | 0.6542 | 0.9612 | 0.9020 | 0.9470 | |
|
| 0.0286 | 3.12 | 450 | 0.2084 | 0.7775 | 0.8052 | 0.7911 | 0.6544 | 0.9624 | 0.8914 | 0.9487 | |
|
| 0.0335 | 3.19 | 460 | 0.2084 | 0.7706 | 0.8144 | 0.7919 | 0.6555 | 0.9621 | 0.8954 | 0.9468 | |
|
| 0.0304 | 3.26 | 470 | 0.2231 | 0.7506 | 0.8280 | 0.7874 | 0.6494 | 0.9604 | 0.9007 | 0.9458 | |
|
| 0.0255 | 3.33 | 480 | 0.2194 | 0.7501 | 0.8348 | 0.7902 | 0.6532 | 0.9608 | 0.9039 | 0.9459 | |
|
| 0.0209 | 3.4 | 490 | 0.2186 | 0.7602 | 0.8300 | 0.7936 | 0.6578 | 0.9618 | 0.9023 | 0.9469 | |
|
| 0.0237 | 3.47 | 500 | 0.2138 | 0.7731 | 0.8196 | 0.7957 | 0.6607 | 0.9628 | 0.8981 | 0.9488 | |
|
| 0.0263 | 3.54 | 510 | 0.2268 | 0.7565 | 0.8313 | 0.7921 | 0.6558 | 0.9614 | 0.9027 | 0.9472 | |
|
| 0.0196 | 3.61 | 520 | 0.2157 | 0.7626 | 0.8242 | 0.7922 | 0.6559 | 0.9617 | 0.8996 | 0.9473 | |
|
| 0.0225 | 3.68 | 530 | 0.2265 | 0.7664 | 0.8185 | 0.7916 | 0.6551 | 0.9619 | 0.8971 | 0.9469 | |
|
| 0.0226 | 3.75 | 540 | 0.2316 | 0.7628 | 0.8139 | 0.7875 | 0.6495 | 0.9611 | 0.8947 | 0.9461 | |
|
| 0.0226 | 3.82 | 550 | 0.2100 | 0.7812 | 0.8130 | 0.7968 | 0.6622 | 0.9633 | 0.8954 | 0.9493 | |
|
| 0.024 | 3.89 | 560 | 0.2213 | 0.7670 | 0.8196 | 0.7924 | 0.6562 | 0.9620 | 0.8977 | 0.9487 | |
|
| 0.0194 | 3.96 | 570 | 0.2227 | 0.7673 | 0.8164 | 0.7911 | 0.6544 | 0.9618 | 0.8962 | 0.9483 | |
|
| 0.0153 | 4.03 | 580 | 0.2296 | 0.7621 | 0.8245 | 0.7921 | 0.6557 | 0.9617 | 0.8997 | 0.9462 | |
|
| 0.0168 | 4.1 | 590 | 0.2362 | 0.7522 | 0.8237 | 0.7863 | 0.6479 | 0.9604 | 0.8987 | 0.9463 | |
|
| 0.0215 | 4.17 | 600 | 0.2413 | 0.7629 | 0.8183 | 0.7896 | 0.6524 | 0.9614 | 0.8968 | 0.9470 | |
|
| 0.0173 | 4.24 | 610 | 0.2411 | 0.7550 | 0.8196 | 0.7859 | 0.6474 | 0.9605 | 0.8969 | 0.9461 | |
|
| 0.0196 | 4.31 | 620 | 0.2315 | 0.7625 | 0.8239 | 0.7920 | 0.6557 | 0.9617 | 0.8995 | 0.9471 | |
|
| 0.0186 | 4.38 | 630 | 0.2593 | 0.7370 | 0.8377 | 0.7841 | 0.6449 | 0.9592 | 0.9043 | 0.9432 | |
|
| 0.0183 | 4.44 | 640 | 0.2368 | 0.7660 | 0.8255 | 0.7947 | 0.6593 | 0.9623 | 0.9005 | 0.9477 | |
|
| 0.0178 | 4.51 | 650 | 0.2326 | 0.7705 | 0.8237 | 0.7962 | 0.6614 | 0.9627 | 0.8999 | 0.9486 | |
|
| 0.017 | 4.58 | 660 | 0.2457 | 0.7448 | 0.8352 | 0.7874 | 0.6493 | 0.9601 | 0.9037 | 0.9454 | |
|
| 0.0175 | 4.65 | 670 | 0.2311 | 0.7694 | 0.8209 | 0.7943 | 0.6588 | 0.9624 | 0.8985 | 0.9485 | |
|
| 0.0163 | 4.72 | 680 | 0.2426 | 0.7696 | 0.8208 | 0.7944 | 0.6589 | 0.9624 | 0.8985 | 0.9484 | |
|
| 0.0163 | 4.79 | 690 | 0.2400 | 0.7527 | 0.8305 | 0.7897 | 0.6525 | 0.9609 | 0.9020 | 0.9462 | |
|
| 0.0122 | 4.86 | 700 | 0.2328 | 0.7656 | 0.8208 | 0.7922 | 0.6559 | 0.9619 | 0.8982 | 0.9476 | |
|
| 0.0205 | 4.93 | 710 | 0.2483 | 0.7474 | 0.8305 | 0.7868 | 0.6485 | 0.9602 | 0.9016 | 0.9452 | |
|
| 0.0154 | 5.0 | 720 | 0.2350 | 0.7612 | 0.8206 | 0.7898 | 0.6526 | 0.9614 | 0.8978 | 0.9472 | |
|
| 0.0155 | 5.07 | 730 | 0.2581 | 0.7570 | 0.8263 | 0.7901 | 0.6531 | 0.9612 | 0.9003 | 0.9463 | |
|
| 0.0137 | 5.14 | 740 | 0.2482 | 0.7592 | 0.8242 | 0.7904 | 0.6534 | 0.9613 | 0.8994 | 0.9472 | |
|
| 0.0124 | 5.21 | 750 | 0.2507 | 0.7639 | 0.8185 | 0.7903 | 0.6533 | 0.9616 | 0.8970 | 0.9466 | |
|
| 0.0121 | 5.28 | 760 | 0.2509 | 0.7586 | 0.8229 | 0.7894 | 0.6521 | 0.9612 | 0.8987 | 0.9470 | |
|
| 0.016 | 5.35 | 770 | 0.2552 | 0.7625 | 0.8173 | 0.7890 | 0.6515 | 0.9613 | 0.8963 | 0.9466 | |
|
| 0.0131 | 5.42 | 780 | 0.2500 | 0.7573 | 0.8231 | 0.7889 | 0.6513 | 0.9610 | 0.8988 | 0.9467 | |
|
| 0.0111 | 5.49 | 790 | 0.2647 | 0.7496 | 0.8312 | 0.7883 | 0.6506 | 0.9605 | 0.9021 | 0.9453 | |
|
| 0.0113 | 5.56 | 800 | 0.2529 | 0.7646 | 0.8226 | 0.7926 | 0.6564 | 0.9619 | 0.8990 | 0.9475 | |
|
| 0.0141 | 5.62 | 810 | 0.2521 | 0.7623 | 0.8230 | 0.7915 | 0.6549 | 0.9616 | 0.8990 | 0.9474 | |
|
| 0.0123 | 5.69 | 820 | 0.2558 | 0.7523 | 0.8282 | 0.7884 | 0.6508 | 0.9607 | 0.9008 | 0.9458 | |
|
| 0.0113 | 5.76 | 830 | 0.2560 | 0.7585 | 0.8208 | 0.7884 | 0.6507 | 0.9610 | 0.8977 | 0.9465 | |
|
| 0.0176 | 5.83 | 840 | 0.2560 | 0.7553 | 0.8263 | 0.7892 | 0.6518 | 0.9609 | 0.9002 | 0.9466 | |
|
| 0.0137 | 5.9 | 850 | 0.2539 | 0.7628 | 0.8233 | 0.7919 | 0.6555 | 0.9617 | 0.8992 | 0.9471 | |
|
| 0.0159 | 5.97 | 860 | 0.2574 | 0.7574 | 0.8246 | 0.7896 | 0.6523 | 0.9611 | 0.8995 | 0.9464 | |
|
| 0.0107 | 6.04 | 870 | 0.2573 | 0.7561 | 0.8257 | 0.7894 | 0.6520 | 0.9610 | 0.8999 | 0.9464 | |
|
| 0.01 | 6.11 | 880 | 0.2562 | 0.7633 | 0.8204 | 0.7908 | 0.6540 | 0.9616 | 0.8978 | 0.9471 | |
|
| 0.0112 | 6.18 | 890 | 0.2574 | 0.7618 | 0.8218 | 0.7907 | 0.6538 | 0.9615 | 0.8984 | 0.9469 | |
|
| 0.0111 | 6.25 | 900 | 0.2563 | 0.7598 | 0.8239 | 0.7906 | 0.6537 | 0.9614 | 0.8993 | 0.9469 | |
|
| 0.0135 | 6.32 | 910 | 0.2558 | 0.7578 | 0.8263 | 0.7905 | 0.6536 | 0.9613 | 0.9003 | 0.9468 | |
|
| 0.0171 | 6.39 | 920 | 0.2554 | 0.7601 | 0.8251 | 0.7913 | 0.6546 | 0.9615 | 0.8999 | 0.9470 | |
|
| 0.0115 | 6.46 | 930 | 0.2563 | 0.7602 | 0.8254 | 0.7915 | 0.6549 | 0.9615 | 0.9001 | 0.9469 | |
|
| 0.0127 | 6.53 | 940 | 0.2561 | 0.7591 | 0.8258 | 0.7910 | 0.6543 | 0.9614 | 0.9002 | 0.9468 | |
|
| 0.0107 | 6.6 | 950 | 0.2563 | 0.7584 | 0.8258 | 0.7906 | 0.6538 | 0.9613 | 0.9001 | 0.9467 | |
|
| 0.0129 | 6.67 | 960 | 0.2567 | 0.7579 | 0.8259 | 0.7905 | 0.6535 | 0.9613 | 0.9002 | 0.9467 | |
|
| 0.012 | 6.74 | 970 | 0.2567 | 0.7580 | 0.8257 | 0.7904 | 0.6534 | 0.9612 | 0.9000 | 0.9467 | |
|
| 0.0141 | 6.81 | 980 | 0.2563 | 0.7587 | 0.8254 | 0.7907 | 0.6538 | 0.9613 | 0.9000 | 0.9468 | |
|
| 0.0116 | 6.88 | 990 | 0.2562 | 0.7588 | 0.8254 | 0.7907 | 0.6539 | 0.9613 | 0.9000 | 0.9468 | |
|
| 0.0097 | 6.94 | 1000 | 0.2562 | 0.7588 | 0.8254 | 0.7907 | 0.6539 | 0.9613 | 0.9000 | 0.9468 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.34.1 |
|
- Pytorch 2.0.1 |
|
- Datasets 2.14.6 |
|
- Tokenizers 0.14.1 |
|
|
