File size: 2,834 Bytes
57cf9e7 bd0b527 57cf9e7 bd0b527 57cf9e7 a2612b7 57cf9e7 2d51696 c69a9c1 8f309e6 8fee06f 91cf863 a21d5d8 744e836 18ec461 29c3129 ff83e8e 072222d 373efea ed8d1c1 0d03b4d e90aba8 291907c 3157a23 139437c 8a2f690 d98894b 30df111 65b5ce0 ab281ce bf64862 271fe42 96ea185 ef04657 b8d2a7b 3a0214b 8c029c4 7f6b4b2 b815c96 3bd96be 42f193b df95a6d 751e69a a2612b7 57cf9e7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
---
license: apache-2.0
base_model: distilgpt2
tags:
- generated_from_keras_callback
model-index:
- name: pippinnie/distilgpt2-finetuned-cyber-readme-v2
results: []
---
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->
# pippinnie/distilgpt2-finetuned-cyber-readme-v2
This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 2.3568
- Validation Loss: 3.0363
- Epoch: 36
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: float32
### Training results
| Train Loss | Validation Loss | Epoch |
|:----------:|:---------------:|:-----:|
| 4.1088 | 3.9258 | 0 |
| 3.9271 | 3.7983 | 1 |
| 3.7845 | 3.6781 | 2 |
| 3.6677 | 3.6006 | 3 |
| 3.5681 | 3.5272 | 4 |
| 3.4803 | 3.4643 | 5 |
| 3.4027 | 3.4068 | 6 |
| 3.3316 | 3.3671 | 7 |
| 3.2666 | 3.3179 | 8 |
| 3.2072 | 3.2817 | 9 |
| 3.1517 | 3.2565 | 10 |
| 3.1007 | 3.2283 | 11 |
| 3.0527 | 3.2051 | 12 |
| 3.0079 | 3.1826 | 13 |
| 2.9651 | 3.1590 | 14 |
| 2.9245 | 3.1529 | 15 |
| 2.8862 | 3.1404 | 16 |
| 2.8493 | 3.1245 | 17 |
| 2.8147 | 3.1075 | 18 |
| 2.7814 | 3.1077 | 19 |
| 2.7497 | 3.1036 | 20 |
| 2.7186 | 3.0859 | 21 |
| 2.6890 | 3.0722 | 22 |
| 2.6608 | 3.0842 | 23 |
| 2.6327 | 3.0561 | 24 |
| 2.6060 | 3.0477 | 25 |
| 2.5804 | 3.0663 | 26 |
| 2.5552 | 3.0479 | 27 |
| 2.5310 | 3.0426 | 28 |
| 2.5066 | 3.0420 | 29 |
| 2.4842 | 3.0671 | 30 |
| 2.4613 | 3.0414 | 31 |
| 2.4394 | 3.0331 | 32 |
| 2.4184 | 3.0449 | 33 |
| 2.3973 | 3.0265 | 34 |
| 2.3764 | 3.0213 | 35 |
| 2.3568 | 3.0363 | 36 |
### Framework versions
- Transformers 4.38.2
- TensorFlow 2.16.1
- Datasets 2.18.0
- Tokenizers 0.15.2
|