File size: 9,050 Bytes
eaf26f3 6b40940 eaf26f3 3f770c0 eaf26f3 6a4e40d eaf26f3 6a4e40d eaf26f3 6a4e40d eaf26f3 6a4e40d eaf26f3 6a4e40d eaf26f3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 |
---
library_name: transformers
tags:
- generated_from_trainer
model-index:
- name: gbert-base-defakts-fake-binary
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# gbert-base-defakts-fake-binary
This Model is finetuned for sequence classification (binary fake-news classification task) on the german DeFaktS-Dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3441
- Accuracy: 0.8526
- F1: 0.8413
- Precision: 0.8545
- Recall: 0.8337
## Model description
This Model is finetuned for sequence classification
### Dataset
Trained on the DeFactS dataset https://github.com/caisa-lab/DeFaktS-Dataset-Disinformaton-Detection, feature catposfake/catneutral to detect fake news
## Intended uses & limitations
Fake news classification
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6
### Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy@de | F1@de | Precision@de | Recall@de | Loss@de |
|:-------------:|:------:|:----:|:---------------:|:-----------:|:------:|:------------:|:---------:|:-------:|
| 0.6236 | 0.0888 | 50 | 0.5162 | 0.7226 | 0.7219 | 0.7377 | 0.7460 | 0.5162 |
| 0.4355 | 0.1776 | 100 | 0.4000 | 0.8201 | 0.8112 | 0.8116 | 0.8107 | 0.4000 |
| 0.3937 | 0.2664 | 150 | 0.3916 | 0.8316 | 0.8181 | 0.8323 | 0.8106 | 0.3917 |
| 0.409 | 0.3552 | 200 | 0.3726 | 0.8331 | 0.8216 | 0.8299 | 0.8163 | 0.3727 |
| 0.3579 | 0.4440 | 250 | 0.3572 | 0.8386 | 0.8285 | 0.8339 | 0.8246 | 0.3573 |
| 0.366 | 0.5329 | 300 | 0.3519 | 0.8456 | 0.8370 | 0.8396 | 0.8349 | 0.3521 |
| 0.3735 | 0.6217 | 350 | 0.4180 | 0.7991 | 0.7976 | 0.8038 | 0.8179 | 0.4180 |
| 0.3849 | 0.7105 | 400 | 0.3605 | 0.8386 | 0.8198 | 0.8592 | 0.8064 | 0.3607 |
| 0.3752 | 0.7993 | 450 | 0.3379 | 0.8581 | 0.8532 | 0.8499 | 0.8586 | 0.3379 |
| 0.3547 | 0.8881 | 500 | 0.3441 | 0.8526 | 0.8413 | 0.8545 | 0.8337 | 0.3442 |
| 0.344 | 0.9769 | 550 | 0.3439 | 0.8461 | 0.8320 | 0.8540 | 0.8218 | 0.3440 |
| 0.2853 | 1.0657 | 600 | 0.3525 | 0.8446 | 0.8386 | 0.8361 | 0.8421 | 0.3526 |
| 0.2505 | 1.1545 | 650 | 0.3427 | 0.8626 | 0.8512 | 0.8686 | 0.8419 | 0.3429 |
| 0.2358 | 1.2433 | 700 | 0.3537 | 0.8626 | 0.8556 | 0.8565 | 0.8547 | 0.3537 |
| 0.257 | 1.3321 | 750 | 0.3258 | 0.8621 | 0.8559 | 0.8548 | 0.8572 | 0.3259 |
| 0.2369 | 1.4210 | 800 | 0.3810 | 0.8551 | 0.8505 | 0.8469 | 0.8571 | 0.3811 |
| 0.248 | 1.5098 | 850 | 0.3576 | 0.8721 | 0.8641 | 0.8701 | 0.8596 | 0.3578 |
| 0.2612 | 1.5986 | 900 | 0.3273 | 0.8686 | 0.8583 | 0.8729 | 0.8500 | 0.3275 |
| 0.2532 | 1.6874 | 950 | 0.3235 | 0.8636 | 0.8567 | 0.8575 | 0.8560 | 0.3236 |
| 0.2225 | 1.7762 | 1000 | 0.3513 | 0.8666 | 0.8567 | 0.8690 | 0.8492 | 0.3515 |
| 0.2497 | 1.8650 | 1050 | 0.3497 | 0.8711 | 0.8625 | 0.8705 | 0.8570 | 0.3498 |
| 0.2291 | 1.9538 | 1100 | 0.3395 | 0.8761 | 0.8685 | 0.8740 | 0.8643 | 0.3396 |
| 0.1675 | 2.0426 | 1150 | 0.3944 | 0.8671 | 0.8583 | 0.8660 | 0.8530 | 0.3946 |
| 0.1182 | 2.1314 | 1200 | 0.4743 | 0.8626 | 0.8532 | 0.8621 | 0.8473 | 0.4746 |
| 0.1453 | 2.2202 | 1250 | 0.4977 | 0.8646 | 0.8531 | 0.8719 | 0.8433 | 0.4981 |
| 0.1288 | 2.3091 | 1300 | 0.4345 | 0.8696 | 0.8631 | 0.8637 | 0.8625 | 0.4347 |
| 0.1399 | 2.3979 | 1350 | 0.4128 | 0.8751 | 0.8665 | 0.8758 | 0.8603 | 0.4132 |
| 0.1384 | 2.4867 | 1400 | 0.3688 | 0.8776 | 0.8713 | 0.8723 | 0.8704 | 0.3690 |
| 0.1292 | 2.5755 | 1450 | 0.4154 | 0.8781 | 0.8710 | 0.8749 | 0.8679 | 0.4157 |
| 0.112 | 2.6643 | 1500 | 0.4399 | 0.8661 | 0.8592 | 0.8603 | 0.8583 | 0.4401 |
| 0.108 | 2.7531 | 1550 | 0.4439 | 0.8731 | 0.8659 | 0.8692 | 0.8631 | 0.4442 |
| 0.1153 | 2.8419 | 1600 | 0.4476 | 0.8676 | 0.8590 | 0.8662 | 0.8539 | 0.4479 |
| 0.1228 | 2.9307 | 1650 | 0.4933 | 0.8691 | 0.8589 | 0.8733 | 0.8506 | 0.4936 |
| 0.0955 | 3.0195 | 1700 | 0.5272 | 0.8806 | 0.8726 | 0.8810 | 0.8668 | 0.5275 |
| 0.0502 | 3.1083 | 1750 | 0.6531 | 0.8661 | 0.8554 | 0.8708 | 0.8468 | 0.6537 |
| 0.0604 | 3.1972 | 1800 | 0.6515 | 0.8721 | 0.8635 | 0.8719 | 0.8578 | 0.6520 |
| 0.068 | 3.2860 | 1850 | 0.6422 | 0.8756 | 0.8656 | 0.8820 | 0.8564 | 0.6427 |
| 0.0505 | 3.3748 | 1900 | 0.6262 | 0.8681 | 0.8606 | 0.8639 | 0.8579 | 0.6266 |
| 0.065 | 3.4636 | 1950 | 0.6342 | 0.8681 | 0.8614 | 0.8623 | 0.8606 | 0.6345 |
| 0.0645 | 3.5524 | 2000 | 0.6472 | 0.8696 | 0.8623 | 0.8653 | 0.8598 | 0.6476 |
| 0.0951 | 3.6412 | 2050 | 0.6048 | 0.8661 | 0.8576 | 0.8641 | 0.8529 | 0.6051 |
| 0.0336 | 3.7300 | 2100 | 0.6603 | 0.8736 | 0.8661 | 0.8705 | 0.8626 | 0.6608 |
| 0.0551 | 3.8188 | 2150 | 0.6932 | 0.8716 | 0.8638 | 0.8689 | 0.8599 | 0.6938 |
| 0.0595 | 3.9076 | 2200 | 0.6379 | 0.8756 | 0.8687 | 0.8715 | 0.8663 | 0.6384 |
| 0.0681 | 3.9964 | 2250 | 0.6327 | 0.8751 | 0.8661 | 0.8774 | 0.8589 | 0.6332 |
| 0.0273 | 4.0853 | 2300 | 0.6414 | 0.8731 | 0.8656 | 0.8701 | 0.8620 | 0.6419 |
| 0.0261 | 4.1741 | 2350 | 0.6590 | 0.8761 | 0.8699 | 0.8705 | 0.8692 | 0.6594 |
| 0.0173 | 4.2629 | 2400 | 0.7341 | 0.8776 | 0.8703 | 0.8750 | 0.8666 | 0.7347 |
| 0.024 | 4.3517 | 2450 | 0.7647 | 0.8706 | 0.8639 | 0.8652 | 0.8626 | 0.7651 |
| 0.0324 | 4.4405 | 2500 | 0.7651 | 0.8741 | 0.8657 | 0.8738 | 0.8601 | 0.7658 |
| 0.0144 | 4.5293 | 2550 | 0.7918 | 0.8691 | 0.8599 | 0.8698 | 0.8535 | 0.7925 |
| 0.0402 | 4.6181 | 2600 | 0.7661 | 0.8691 | 0.8604 | 0.8684 | 0.8549 | 0.7667 |
| 0.0311 | 4.7069 | 2650 | 0.7688 | 0.8706 | 0.8619 | 0.8701 | 0.8564 | 0.7694 |
| 0.0127 | 4.7957 | 2700 | 0.7880 | 0.8691 | 0.8607 | 0.8675 | 0.8558 | 0.7886 |
| 0.03 | 4.8845 | 2750 | 0.7616 | 0.8736 | 0.8672 | 0.8680 | 0.8665 | 0.7620 |
| 0.0431 | 4.9734 | 2800 | 0.7842 | 0.8716 | 0.8631 | 0.8710 | 0.8576 | 0.7848 |
| 0.0243 | 5.0622 | 2850 | 0.7620 | 0.8701 | 0.8612 | 0.8704 | 0.8550 | 0.7626 |
| 0.0135 | 5.1510 | 2900 | 0.7799 | 0.8711 | 0.8624 | 0.8710 | 0.8565 | 0.7805 |
| 0.0177 | 5.2398 | 2950 | 0.7644 | 0.8746 | 0.8672 | 0.8716 | 0.8637 | 0.7649 |
| 0.0126 | 5.3286 | 3000 | 0.7826 | 0.8736 | 0.8656 | 0.8721 | 0.8608 | 0.7832 |
| 0.011 | 5.4174 | 3050 | 0.7951 | 0.8761 | 0.8681 | 0.8751 | 0.8631 | 0.7957 |
| 0.0214 | 5.5062 | 3100 | 0.7953 | 0.8741 | 0.8657 | 0.8738 | 0.8601 | 0.7960 |
| 0.0099 | 5.5950 | 3150 | 0.7855 | 0.8746 | 0.8667 | 0.8729 | 0.8621 | 0.7861 |
| 0.0193 | 5.6838 | 3200 | 0.7967 | 0.8746 | 0.8661 | 0.8747 | 0.8603 | 0.7974 |
| 0.0171 | 5.7726 | 3250 | 0.7956 | 0.8751 | 0.8669 | 0.8744 | 0.8616 | 0.7962 |
| 0.013 | 5.8615 | 3300 | 0.7972 | 0.8741 | 0.8658 | 0.8736 | 0.8604 | 0.7978 |
| 0.0176 | 5.9503 | 3350 | 0.8003 | 0.8736 | 0.8651 | 0.8734 | 0.8595 | 0.8009 |
### Framework versions
- Transformers 4.45.2
- Pytorch 2.3.1+cu121
- Tokenizers 0.20.3
|