Hugging Face's logo

Sahyus
/
roberta-large-squad2-finetuned-dtc

Question Answering
Transformers
TensorFlow
roberta
generated_from_keras_callback
Inference Endpoints
Model card Files Community
roberta-large-squad2-finetuned-dtc / README.md
Sahyus's picture
Sahyus
Upload TFRobertaForQuestionAnswering
2f7a9ae
preview code
|
raw
history blame contribute delete
11.8 kB
metadata
license: cc-by-4.0
base_model: deepset/roberta-large-squad2
tags:
  - generated_from_keras_callback
model-index:
  - name: roberta-large-squad2-finetuned-dtc
    results: []

roberta-large-squad2-finetuned-dtc

This model is a fine-tuned version of deepset/roberta-large-squad2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 1.9389
  • Train End Logits Loss: 1.1432
  • Train Start Logits Loss: 0.7957
  • Train End Logits Acc: 0.7392
  • Train Start Logits Acc: 0.8093
  • Validation Loss: 3.7259
  • Validation End Logits Loss: 1.8885
  • Validation Start Logits Loss: 1.8374
  • Validation End Logits Acc: 0.6312
  • Validation Start Logits Acc: 0.7221
  • Epoch: 36

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2.4e-05, 'decay_steps': 21400, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.03}
  • training_precision: float32

Training results

Train Loss Train End Logits Loss Train Start Logits Loss Train End Logits Acc Train Start Logits Acc Validation Loss Validation End Logits Loss Validation Start Logits Loss Validation End Logits Acc Validation Start Logits Acc Epoch
5.8888 3.0592 2.8296 0.5456 0.5406 4.8715 2.6861 2.1854 0.6130 0.6182 0
5.0000 2.7063 2.2937 0.5809 0.5762 4.0680 2.3593 1.7087 0.6208 0.6000 1
4.7529 2.5952 2.1576 0.5929 0.5862 4.5767 2.7450 1.8317 0.6208 0.6156 2
4.6181 2.5511 2.0670 0.5984 0.5873 3.9828 2.4125 1.5703 0.6208 0.6052 3
4.4766 2.4920 1.9846 0.6019 0.5946 3.7862 2.2460 1.5402 0.6208 0.5922 4
4.5692 2.5720 1.9972 0.6081 0.6066 3.6069 2.1558 1.4511 0.6208 0.6052 5
4.3098 2.4382 1.8716 0.6016 0.5987 3.2741 1.9275 1.3466 0.6208 0.6364 6
3.8906 2.2240 1.6666 0.6165 0.6256 3.3856 1.9718 1.4138 0.6156 0.6052 7
3.7711 2.1773 1.5939 0.6154 0.6317 3.4381 1.7916 1.6465 0.6182 0.4805 8
3.6331 2.1149 1.5182 0.6177 0.6460 3.0055 1.6855 1.3200 0.5273 0.6338 9
3.4683 2.0212 1.4471 0.6168 0.6545 3.3422 1.7875 1.5547 0.4805 0.5325 10
3.3695 1.9567 1.4129 0.6183 0.6618 2.8283 1.5488 1.2795 0.5455 0.6286 11
3.3125 1.9344 1.3781 0.6215 0.6647 2.7086 1.5124 1.1962 0.5636 0.6338 12
3.2580 1.9282 1.3298 0.6390 0.6852 3.0502 1.7520 1.2982 0.6156 0.6623 13
3.2814 1.9478 1.3336 0.6294 0.6711 2.5437 1.4591 1.0846 0.5948 0.6727 14
3.1027 1.8305 1.2721 0.6370 0.6893 3.0537 1.6897 1.3640 0.5481 0.5922 15
2.7670 1.6628 1.1042 0.6583 0.7217 2.4372 1.3791 1.0581 0.6519 0.6961 16
2.7880 1.6975 1.0905 0.6583 0.7339 2.2441 1.2735 0.9706 0.7039 0.7299 17
2.7786 1.6524 1.1262 0.6606 0.7225 2.6408 1.4267 1.2141 0.6701 0.6831 18
2.4685 1.4862 0.9823 0.6741 0.7447 2.7726 1.5947 1.1779 0.6338 0.6909 19
2.4204 1.4523 0.9682 0.6814 0.7538 2.1115 1.1877 0.9238 0.7429 0.7714 20
2.2158 1.3472 0.8686 0.6939 0.7707 2.2647 1.2382 1.0266 0.7143 0.7532 21
2.0138 1.2461 0.7676 0.7109 0.7994 2.1425 1.1617 0.9808 0.7455 0.7558 22
2.0038 1.2585 0.7453 0.7129 0.8008 1.8952 0.9984 0.8968 0.7688 0.7558 23
1.8391 1.1600 0.6791 0.7231 0.8186 2.4242 1.3208 1.1034 0.7013 0.7039 24
1.7792 1.1060 0.6732 0.7389 0.8248 1.8800 1.0211 0.8588 0.7792 0.7818 25
1.6690 1.0636 0.6054 0.7462 0.8367 2.2503 1.2198 1.0305 0.7325 0.7506 26
1.6197 1.0327 0.5870 0.7591 0.8452 1.9393 0.9581 0.9812 0.7974 0.8052 27
1.5335 0.9795 0.5540 0.7652 0.8595 2.2046 1.1750 1.0296 0.7688 0.7870 28
1.4563 0.9314 0.5249 0.7751 0.8621 1.9638 1.0204 0.9434 0.7403 0.7792 29
1.3903 0.9049 0.4854 0.7772 0.8683 2.2657 1.1569 1.1088 0.7636 0.7896 30
1.3534 0.8813 0.4720 0.7859 0.8744 1.9620 0.9779 0.9840 0.7688 0.7740 31
1.4848 0.9444 0.5405 0.7684 0.8563 2.3368 1.1941 1.1427 0.7299 0.7688 32
1.5092 0.9534 0.5558 0.7550 0.8461 2.1233 1.0956 1.0277 0.7610 0.7740 33
1.4016 0.8789 0.5227 0.7751 0.8624 2.4886 1.2593 1.2294 0.7403 0.7844 34
1.8007 1.0509 0.7498 0.7520 0.8183 2.5730 1.3045 1.2686 0.7195 0.7481 35
1.9389 1.1432 0.7957 0.7392 0.8093 3.7259 1.8885 1.8374 0.6312 0.7221 36

Framework versions

  • Transformers 4.36.2
  • TensorFlow 2.14.0
  • Datasets 2.16.1
  • Tokenizers 0.15.0