--- license: apache-2.0 base_model: bert-base-uncased tags: - generated_from_trainer metrics: - accuracy model-index: - name: bert-base-uncased-finetuned-github_cybersecurity_READMEs results: [] --- # bert-base-uncased-finetuned-github_cybersecurity_READMEs This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 2.2291 - Accuracy: 0.6479 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 32 - eval_batch_size: 32 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 128 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 1000 - num_epochs: 100 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:----:|:---------------:|:--------:| | No log | 0.97 | 14 | 4.1856 | 0.4305 | | No log | 2.0 | 29 | 4.3178 | 0.4090 | | No log | 2.97 | 43 | 4.0734 | 0.4342 | | No log | 4.0 | 58 | 4.0470 | 0.4332 | | No log | 4.97 | 72 | 4.0668 | 0.4270 | | No log | 6.0 | 87 | 3.9068 | 0.4390 | | No log | 6.97 | 101 | 3.8466 | 0.4468 | | No log | 8.0 | 116 | 3.8330 | 0.4535 | | No log | 8.97 | 130 | 3.7238 | 0.4516 | | No log | 10.0 | 145 | 3.8113 | 0.4446 | | No log | 10.97 | 159 | 3.6681 | 0.4607 | | No log | 12.0 | 174 | 3.5627 | 0.4679 | | No log | 12.97 | 188 | 3.4540 | 0.4794 | | No log | 14.0 | 203 | 3.5997 | 0.4707 | | No log | 14.97 | 217 | 3.4362 | 0.4860 | | No log | 16.0 | 232 | 3.5471 | 0.4740 | | No log | 16.97 | 246 | 3.4968 | 0.4803 | | No log | 18.0 | 261 | 3.2938 | 0.4985 | | No log | 18.97 | 275 | 3.4207 | 0.4765 | | No log | 20.0 | 290 | 3.3869 | 0.4970 | | No log | 20.97 | 304 | 3.3062 | 0.5012 | | No log | 22.0 | 319 | 3.3184 | 0.4917 | | No log | 22.97 | 333 | 3.2132 | 0.5136 | | No log | 24.0 | 348 | 3.2027 | 0.5074 | | No log | 24.97 | 362 | 3.3251 | 0.4923 | | No log | 26.0 | 377 | 3.1569 | 0.5108 | | No log | 26.97 | 391 | 3.0947 | 0.5194 | | No log | 28.0 | 406 | 3.0470 | 0.5206 | | No log | 28.97 | 420 | 3.0662 | 0.5182 | | No log | 30.0 | 435 | 3.0845 | 0.5191 | | No log | 30.97 | 449 | 3.0681 | 0.5219 | | No log | 32.0 | 464 | 2.9902 | 0.5263 | | No log | 32.97 | 478 | 2.8970 | 0.5448 | | No log | 34.0 | 493 | 2.9269 | 0.5341 | | 3.629 | 34.97 | 507 | 2.8605 | 0.5519 | | 3.629 | 36.0 | 522 | 2.8657 | 0.5431 | | 3.629 | 36.97 | 536 | 2.9391 | 0.5407 | | 3.629 | 38.0 | 551 | 2.8960 | 0.5437 | | 3.629 | 38.97 | 565 | 2.8819 | 0.5466 | | 3.629 | 40.0 | 580 | 2.7555 | 0.5633 | | 3.629 | 40.97 | 594 | 2.7425 | 0.5555 | | 3.629 | 42.0 | 609 | 2.7960 | 0.5615 | | 3.629 | 42.97 | 623 | 2.7382 | 0.5630 | | 3.629 | 44.0 | 638 | 2.7967 | 0.5580 | | 3.629 | 44.97 | 652 | 2.6611 | 0.5781 | | 3.629 | 46.0 | 667 | 2.6877 | 0.5722 | | 3.629 | 46.97 | 681 | 2.7917 | 0.5609 | | 3.629 | 48.0 | 696 | 2.7029 | 0.5696 | | 3.629 | 48.97 | 710 | 2.7408 | 0.5618 | | 3.629 | 50.0 | 725 | 2.6450 | 0.5772 | | 3.629 | 50.97 | 739 | 2.5569 | 0.5883 | | 3.629 | 52.0 | 754 | 2.6646 | 0.5795 | | 3.629 | 52.97 | 768 | 2.6803 | 0.5729 | | 3.629 | 54.0 | 783 | 2.6233 | 0.5847 | | 3.629 | 54.97 | 797 | 2.6027 | 0.5842 | | 3.629 | 56.0 | 812 | 2.4090 | 0.6034 | | 3.629 | 56.97 | 826 | 2.4978 | 0.6011 | | 3.629 | 58.0 | 841 | 2.5106 | 0.5944 | | 3.629 | 58.97 | 855 | 2.5039 | 0.5912 | | 3.629 | 60.0 | 870 | 2.5792 | 0.5824 | | 3.629 | 60.97 | 884 | 2.4764 | 0.6065 | | 3.629 | 62.0 | 899 | 2.5348 | 0.6036 | | 3.629 | 62.97 | 913 | 2.5338 | 0.6022 | | 3.629 | 64.0 | 928 | 2.4646 | 0.6130 | | 3.629 | 64.97 | 942 | 2.4532 | 0.6066 | | 3.629 | 66.0 | 957 | 2.4526 | 0.6073 | | 3.629 | 66.97 | 971 | 2.5369 | 0.5992 | | 3.629 | 68.0 | 986 | 2.4170 | 0.6181 | | 2.5556 | 68.97 | 1000 | 2.4493 | 0.6078 | | 2.5556 | 70.0 | 1015 | 2.3939 | 0.6159 | | 2.5556 | 70.97 | 1029 | 2.4793 | 0.6049 | | 2.5556 | 72.0 | 1044 | 2.3225 | 0.6286 | | 2.5556 | 72.97 | 1058 | 2.3551 | 0.6212 | | 2.5556 | 74.0 | 1073 | 2.4702 | 0.6075 | | 2.5556 | 74.97 | 1087 | 2.3489 | 0.6311 | | 2.5556 | 76.0 | 1102 | 2.3455 | 0.6198 | | 2.5556 | 76.97 | 1116 | 2.4500 | 0.6145 | | 2.5556 | 78.0 | 1131 | 2.3223 | 0.6332 | | 2.5556 | 78.97 | 1145 | 2.4375 | 0.6065 | | 2.5556 | 80.0 | 1160 | 2.2743 | 0.6291 | | 2.5556 | 80.97 | 1174 | 2.3255 | 0.6295 | | 2.5556 | 82.0 | 1189 | 2.3785 | 0.6237 | | 2.5556 | 82.97 | 1203 | 2.2722 | 0.6344 | | 2.5556 | 84.0 | 1218 | 2.2392 | 0.6407 | | 2.5556 | 84.97 | 1232 | 2.2322 | 0.6361 | | 2.5556 | 86.0 | 1247 | 2.2206 | 0.6496 | | 2.5556 | 86.97 | 1261 | 2.2419 | 0.6345 | | 2.5556 | 88.0 | 1276 | 2.1919 | 0.6492 | | 2.5556 | 88.97 | 1290 | 2.2616 | 0.6433 | | 2.5556 | 90.0 | 1305 | 2.2227 | 0.6417 | | 2.5556 | 90.97 | 1319 | 2.2847 | 0.6447 | | 2.5556 | 92.0 | 1334 | 2.2916 | 0.6339 | | 2.5556 | 92.97 | 1348 | 2.2684 | 0.6410 | | 2.5556 | 94.0 | 1363 | 2.2432 | 0.6440 | | 2.5556 | 94.97 | 1377 | 2.2510 | 0.6462 | | 2.5556 | 96.0 | 1392 | 2.2970 | 0.6363 | | 2.5556 | 96.55 | 1400 | 2.2197 | 0.6423 | ### Framework versions - Transformers 4.40.0.dev0 - Pytorch 2.2.1+cu121 - Datasets 2.18.0 - Tokenizers 0.15.2