nbbert_ED / README.md
yemen2016's picture
End of training
b744ba8 verified
metadata
library_name: transformers
license: cc-by-4.0
base_model: NbAiLab/nb-bert-base
tags:
  - generated_from_trainer
model-index:
  - name: nbbert_ED
    results: []

nbbert_ED

This model is a fine-tuned version of NbAiLab/nb-bert-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9955
  • F1-score: 0.8361

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss F1-score
No log 1.0 69 0.6947 0.4209
No log 2.0 138 0.8251 0.6436
No log 3.0 207 0.6215 0.7587
No log 4.0 276 0.5942 0.7622
No log 5.0 345 0.6512 0.7622
No log 6.0 414 0.5853 0.7855
No log 7.0 483 1.1781 0.6619
0.4341 8.0 552 0.9684 0.7596
0.4341 9.0 621 0.8108 0.7951
0.4341 10.0 690 0.9732 0.7849
0.4341 11.0 759 0.8429 0.8276
0.4341 12.0 828 1.1912 0.7576
0.4341 13.0 897 1.0208 0.8115
0.4341 14.0 966 0.9234 0.8197
0.1528 15.0 1035 0.8931 0.8357
0.1528 16.0 1104 1.1005 0.8025
0.1528 17.0 1173 0.9808 0.8279
0.1528 18.0 1242 1.0438 0.8195
0.1528 19.0 1311 1.0193 0.8197
0.1528 20.0 1380 0.9955 0.8361

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.4.0+cu121
  • Datasets 2.21.0
  • Tokenizers 0.19.1