Hugging Face's logo

sickcell
/
distilgpt2-finetuned-github_cybersecurity_READMEs

Text Generation
Transformers
TensorBoard
Safetensors
gpt2
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
sickcell69 commited on
Commit
8893a44
•
1 Parent(s): b4a48dd

End of training

Browse files
Files changed (12) hide show
  1. README.md +18 -10
  2. model.safetensors +1 -1
  3. runs/Apr13_18-16-53_DESKTOP-7EBBP1S/events.out.tfevents.1713003413.DESKTOP-7EBBP1S.223104.2 +2 -2
  4. runs/Apr13_18-16-53_DESKTOP-7EBBP1S/events.out.tfevents.1713004552.DESKTOP-7EBBP1S.223104.3 +3 -0
  5. runs/Apr13_19-14-31_DESKTOP-7EBBP1S/events.out.tfevents.1713006873.DESKTOP-7EBBP1S.225414.0 +3 -0
  6. runs/Apr13_19-16-30_DESKTOP-7EBBP1S/events.out.tfevents.1713006993.DESKTOP-7EBBP1S.225414.1 +3 -0
  7. runs/Apr13_19-16-30_DESKTOP-7EBBP1S/events.out.tfevents.1713007194.DESKTOP-7EBBP1S.225414.2 +3 -0
  8. runs/Apr13_19-27-28_DESKTOP-7EBBP1S/events.out.tfevents.1713007649.DESKTOP-7EBBP1S.226230.2 +3 -0
  9. runs/Apr13_19-27-28_DESKTOP-7EBBP1S/events.out.tfevents.1713007732.DESKTOP-7EBBP1S.226230.3 +3 -0
  10. runs/Apr13_19-41-38_DESKTOP-7EBBP1S/events.out.tfevents.1713008498.DESKTOP-7EBBP1S.227770.0 +3 -0
  11. runs/Apr13_19-41-38_DESKTOP-7EBBP1S/events.out.tfevents.1713008551.DESKTOP-7EBBP1S.227770.1 +3 -0
  12. training_args.bin +1 -1
README.md CHANGED
@@ -3,6 +3,8 @@ license: apache-2.0
3
  base_model: distilgpt2
4
  tags:
5
  - generated_from_trainer
 
 
6
  model-index:
7
  - name: distilgpt2-finetuned-github_cybersecurity_READMEs
8
  results: []
@@ -15,7 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 2.5260
 
19
 
20
  ## Model description
21
 
@@ -34,21 +37,26 @@ More information needed
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
- - learning_rate: 2e-05
38
- - train_batch_size: 8
39
- - eval_batch_size: 8
40
  - seed: 42
 
 
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
- - num_epochs: 3.0
 
44
 
45
  ### Training results
46
 
47
- | Training Loss | Epoch | Step | Validation Loss |
48
- |:-------------:|:-----:|:-----:|:---------------:|
49
- | 2.4374 | 1.0 | 23960 | 2.6365 |
50
- | 2.3011 | 2.0 | 47920 | 2.5576 |
51
- | 2.2825 | 3.0 | 71880 | 2.5260 |
 
 
52
 
53
 
54
  ### Framework versions
 
3
  base_model: distilgpt2
4
  tags:
5
  - generated_from_trainer
6
+ metrics:
7
+ - accuracy
8
  model-index:
9
  - name: distilgpt2-finetuned-github_cybersecurity_READMEs
10
  results: []
 
17
 
18
  This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 5.4529
21
+ - Accuracy: 0.0699
22
 
23
  ## Model description
24
 
 
37
  ### Training hyperparameters
38
 
39
  The following hyperparameters were used during training:
40
+ - learning_rate: 3e-05
41
+ - train_batch_size: 32
42
+ - eval_batch_size: 32
43
  - seed: 42
44
+ - gradient_accumulation_steps: 4
45
+ - total_train_batch_size: 128
46
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
47
  - lr_scheduler_type: linear
48
+ - lr_scheduler_warmup_steps: 1000
49
+ - num_epochs: 5
50
 
51
  ### Training results
52
 
53
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
54
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|
55
+ | No log | 0.97 | 14 | 5.6664 | 0.0648 |
56
+ | No log | 2.0 | 29 | 5.6299 | 0.0653 |
57
+ | No log | 2.97 | 43 | 5.5792 | 0.0660 |
58
+ | No log | 4.0 | 58 | 5.5133 | 0.0664 |
59
+ | No log | 4.83 | 70 | 5.4529 | 0.0699 |
60
 
61
 
62
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:34d4d98e61e3e181d52b603ee38384bc213f2314bbb10ef9128d546726147922
3
  size 327657928
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcbd8d52304cab1c8974cd089e269f2f6a83616e576aecb272ab4ee24ff7a60e
3
  size 327657928
runs/Apr13_18-16-53_DESKTOP-7EBBP1S/events.out.tfevents.1713003413.DESKTOP-7EBBP1S.223104.2 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:078eb80e06da547c32e52876691c3f2a3b24c93d984c353b8950cf6e589290c5
3
- size 16187
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:736a0e9ee7307790f849dc48b50ace4d7ba1db247efba46e4796c6a66dfd8599
3
+ size 37101
runs/Apr13_18-16-53_DESKTOP-7EBBP1S/events.out.tfevents.1713004552.DESKTOP-7EBBP1S.223104.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b542c40cf2ab4de658080a01c1f43cce0e76c6ba540e736afdeb44b3725f23af
3
+ size 411
runs/Apr13_19-14-31_DESKTOP-7EBBP1S/events.out.tfevents.1713006873.DESKTOP-7EBBP1S.225414.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:952e508f33752e442fd2c868ad7b64df7973393e0f09d1d87833d661caafd988
3
+ size 5358
runs/Apr13_19-16-30_DESKTOP-7EBBP1S/events.out.tfevents.1713006993.DESKTOP-7EBBP1S.225414.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:53bcb31eba97ec19df2a079b37882c3cfb1ef778f513577cf806a2acfc58f429
3
+ size 6973
runs/Apr13_19-16-30_DESKTOP-7EBBP1S/events.out.tfevents.1713007194.DESKTOP-7EBBP1S.225414.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0014b163df471e0c47a006b7f27726672a41d559eaece6ef2b338b8e8ea7b7a1
3
+ size 405
runs/Apr13_19-27-28_DESKTOP-7EBBP1S/events.out.tfevents.1713007649.DESKTOP-7EBBP1S.226230.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de1c55e729854e3e7b1406266b309ae52bb79a56dac370170b4fef34da630f60
3
+ size 6973
runs/Apr13_19-27-28_DESKTOP-7EBBP1S/events.out.tfevents.1713007732.DESKTOP-7EBBP1S.226230.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45961019cc44ed68cae21ba82904658542fcf8aff9f8945fcee7146161105854
3
+ size 405
runs/Apr13_19-41-38_DESKTOP-7EBBP1S/events.out.tfevents.1713008498.DESKTOP-7EBBP1S.227770.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2954de8c0567a79db494ce161be4e8c105b5e9cc070921e7f66ddca36320b22
3
+ size 6973
runs/Apr13_19-41-38_DESKTOP-7EBBP1S/events.out.tfevents.1713008551.DESKTOP-7EBBP1S.227770.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eedf60c40ef07f7930b1d36336324cfa6a5f5cb8661ffa92f5d332bac149a1ca
3
+ size 405
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cb7ac9c7ac5c0a261d3a78d7d343e8188d0c6e93e7f4d53114231dce424b62a6
3
  size 4984
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7cdaa9837824bb2f1bd1cf858f610f017082ebba79b1529db844784ec318bbd
3
  size 4984