Edit model card

danskbert_ED

This model is a fine-tuned version of vesteinn/DanskBERT on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7439
  • F1-score: 0.8339

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss F1-score
No log 1.0 69 0.6389 0.8014
No log 2.0 138 0.6085 0.7654
No log 3.0 207 0.7439 0.8339
No log 4.0 276 0.8447 0.8273
No log 5.0 345 0.9992 0.8193
No log 6.0 414 1.4570 0.7775
No log 7.0 483 1.4951 0.8029
0.2302 8.0 552 1.7546 0.7769
0.2302 9.0 621 1.5325 0.8115
0.2302 10.0 690 1.6252 0.8033
0.2302 11.0 759 1.5428 0.8197
0.2302 12.0 828 1.5487 0.8278
0.2302 13.0 897 1.5563 0.8195
0.2302 14.0 966 1.5723 0.8195
0.0017 15.0 1035 1.5878 0.8276
0.0017 16.0 1104 1.6001 0.8276
0.0017 17.0 1173 1.6105 0.8276
0.0017 18.0 1242 1.6195 0.8276
0.0017 19.0 1311 1.6236 0.8276
0.0017 20.0 1380 1.6166 0.8278

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.4.0+cu121
  • Datasets 2.21.0
  • Tokenizers 0.19.1
Downloads last month
7
Safetensors
Model size
124M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for yemen2016/danskbert_ED

Base model

vesteinn/DanskBERT
Finetuned
(13)
this model