Commit
•
59b81cc
1
Parent(s):
8bc41ba
End of training
Browse files- README.md +0 -88
- adapter_model.safetensors +1 -1
README.md
CHANGED
|
@@ -15,8 +15,6 @@ should probably proofread and complete it, then remove this comment. -->
|
|
| 15 |
# tinyllama-v1-finetune
|
| 16 |
|
| 17 |
This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on the None dataset.
|
| 18 |
-
It achieves the following results on the evaluation set:
|
| 19 |
-
- Loss: 0.5205
|
| 20 |
|
| 21 |
## Model description
|
| 22 |
|
|
@@ -46,92 +44,6 @@ The following hyperparameters were used during training:
|
|
| 46 |
- training_steps: 400
|
| 47 |
- mixed_precision_training: Native AMP
|
| 48 |
|
| 49 |
-
### Training results
|
| 50 |
-
|
| 51 |
-
| Training Loss | Epoch | Step | Validation Loss |
|
| 52 |
-
|:-------------:|:-----:|:----:|:---------------:|
|
| 53 |
-
| 2.7664 | 0.04 | 5 | 2.7807 |
|
| 54 |
-
| 2.7326 | 0.08 | 10 | 2.7249 |
|
| 55 |
-
| 2.6955 | 0.12 | 15 | 2.6739 |
|
| 56 |
-
| 2.643 | 0.16 | 20 | 2.6085 |
|
| 57 |
-
| 2.6056 | 0.2 | 25 | 2.5554 |
|
| 58 |
-
| 2.5018 | 0.24 | 30 | 2.4923 |
|
| 59 |
-
| 2.4587 | 0.28 | 35 | 2.4204 |
|
| 60 |
-
| 2.3997 | 0.32 | 40 | 2.3484 |
|
| 61 |
-
| 2.3142 | 0.36 | 45 | 2.2673 |
|
| 62 |
-
| 2.2507 | 0.4 | 50 | 2.1903 |
|
| 63 |
-
| 2.1405 | 0.43 | 55 | 2.1071 |
|
| 64 |
-
| 2.0471 | 0.47 | 60 | 2.0212 |
|
| 65 |
-
| 1.9957 | 0.51 | 65 | 1.9385 |
|
| 66 |
-
| 1.9007 | 0.55 | 70 | 1.8448 |
|
| 67 |
-
| 1.8046 | 0.59 | 75 | 1.7501 |
|
| 68 |
-
| 1.691 | 0.63 | 80 | 1.6495 |
|
| 69 |
-
| 1.5957 | 0.67 | 85 | 1.5484 |
|
| 70 |
-
| 1.53 | 0.71 | 90 | 1.4608 |
|
| 71 |
-
| 1.4161 | 0.75 | 95 | 1.3669 |
|
| 72 |
-
| 1.3409 | 0.79 | 100 | 1.2772 |
|
| 73 |
-
| 1.239 | 0.83 | 105 | 1.1833 |
|
| 74 |
-
| 1.1591 | 0.87 | 110 | 1.1122 |
|
| 75 |
-
| 1.0729 | 0.91 | 115 | 1.0276 |
|
| 76 |
-
| 1.0237 | 0.95 | 120 | 0.9615 |
|
| 77 |
-
| 0.9293 | 0.99 | 125 | 0.9021 |
|
| 78 |
-
| 0.891 | 1.03 | 130 | 0.8531 |
|
| 79 |
-
| 0.8365 | 1.07 | 135 | 0.8155 |
|
| 80 |
-
| 0.7876 | 1.11 | 140 | 0.7898 |
|
| 81 |
-
| 0.7821 | 1.15 | 145 | 0.7669 |
|
| 82 |
-
| 0.7392 | 1.19 | 150 | 0.7516 |
|
| 83 |
-
| 0.77 | 1.23 | 155 | 0.7397 |
|
| 84 |
-
| 0.7088 | 1.26 | 160 | 0.7226 |
|
| 85 |
-
| 0.7246 | 1.3 | 165 | 0.7101 |
|
| 86 |
-
| 0.7007 | 1.34 | 170 | 0.6960 |
|
| 87 |
-
| 0.6667 | 1.38 | 175 | 0.6797 |
|
| 88 |
-
| 0.6898 | 1.42 | 180 | 0.6666 |
|
| 89 |
-
| 0.6608 | 1.46 | 185 | 0.6599 |
|
| 90 |
-
| 0.6526 | 1.5 | 190 | 0.6451 |
|
| 91 |
-
| 0.6078 | 1.54 | 195 | 0.6350 |
|
| 92 |
-
| 0.6336 | 1.58 | 200 | 0.6248 |
|
| 93 |
-
| 0.6074 | 1.62 | 205 | 0.6167 |
|
| 94 |
-
| 0.6114 | 1.66 | 210 | 0.6131 |
|
| 95 |
-
| 0.575 | 1.7 | 215 | 0.6041 |
|
| 96 |
-
| 0.5933 | 1.74 | 220 | 0.5981 |
|
| 97 |
-
| 0.5983 | 1.78 | 225 | 0.5910 |
|
| 98 |
-
| 0.5907 | 1.82 | 230 | 0.5845 |
|
| 99 |
-
| 0.5853 | 1.86 | 235 | 0.5801 |
|
| 100 |
-
| 0.5881 | 1.9 | 240 | 0.5749 |
|
| 101 |
-
| 0.5613 | 1.94 | 245 | 0.5700 |
|
| 102 |
-
| 0.5852 | 1.98 | 250 | 0.5696 |
|
| 103 |
-
| 0.5781 | 2.02 | 255 | 0.5652 |
|
| 104 |
-
| 0.5812 | 2.06 | 260 | 0.5609 |
|
| 105 |
-
| 0.5677 | 2.09 | 265 | 0.5576 |
|
| 106 |
-
| 0.5544 | 2.13 | 270 | 0.5541 |
|
| 107 |
-
| 0.536 | 2.17 | 275 | 0.5504 |
|
| 108 |
-
| 0.5283 | 2.21 | 280 | 0.5487 |
|
| 109 |
-
| 0.5326 | 2.25 | 285 | 0.5454 |
|
| 110 |
-
| 0.568 | 2.29 | 290 | 0.5402 |
|
| 111 |
-
| 0.5448 | 2.33 | 295 | 0.5395 |
|
| 112 |
-
| 0.5581 | 2.37 | 300 | 0.5377 |
|
| 113 |
-
| 0.5406 | 2.41 | 305 | 0.5355 |
|
| 114 |
-
| 0.4996 | 2.45 | 310 | 0.5333 |
|
| 115 |
-
| 0.5243 | 2.49 | 315 | 0.5346 |
|
| 116 |
-
| 0.5591 | 2.53 | 320 | 0.5312 |
|
| 117 |
-
| 0.5122 | 2.57 | 325 | 0.5297 |
|
| 118 |
-
| 0.5426 | 2.61 | 330 | 0.5290 |
|
| 119 |
-
| 0.4955 | 2.65 | 335 | 0.5290 |
|
| 120 |
-
| 0.5531 | 2.69 | 340 | 0.5273 |
|
| 121 |
-
| 0.5147 | 2.73 | 345 | 0.5278 |
|
| 122 |
-
| 0.5195 | 2.77 | 350 | 0.5246 |
|
| 123 |
-
| 0.5268 | 2.81 | 355 | 0.5254 |
|
| 124 |
-
| 0.5284 | 2.85 | 360 | 0.5236 |
|
| 125 |
-
| 0.5272 | 2.89 | 365 | 0.5224 |
|
| 126 |
-
| 0.5053 | 2.92 | 370 | 0.5240 |
|
| 127 |
-
| 0.528 | 2.96 | 375 | 0.5200 |
|
| 128 |
-
| 0.517 | 3.0 | 380 | 0.5204 |
|
| 129 |
-
| 0.5409 | 3.04 | 385 | 0.5206 |
|
| 130 |
-
| 0.5204 | 3.08 | 390 | 0.5192 |
|
| 131 |
-
| 0.52 | 3.12 | 395 | 0.5200 |
|
| 132 |
-
| 0.5142 | 3.16 | 400 | 0.5205 |
|
| 133 |
-
|
| 134 |
-
|
| 135 |
### Framework versions
|
| 136 |
|
| 137 |
- PEFT 0.8.1
|
|
|
|
| 15 |
# tinyllama-v1-finetune
|
| 16 |
|
| 17 |
This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on the None dataset.
|
|
|
|
|
|
|
| 18 |
|
| 19 |
## Model description
|
| 20 |
|
|
|
|
| 44 |
- training_steps: 400
|
| 45 |
- mixed_precision_training: Native AMP
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
### Framework versions
|
| 48 |
|
| 49 |
- PEFT 0.8.1
|
adapter_model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4517152
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6fbfed0cf2bdce82c539c14ea965f3a15dc94c3d60490bfe765ff35f84a8c0c3
|
| 3 |
size 4517152
|