End of training

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [LoftQ/Meta-Llama-3-8B-Instruct-4bit-64rank](https://huggingface.co/LoftQ/Meta-Llama-3-8B-Instruct-4bit-64rank) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.9675
+- Loss: 0.8530
 
 ## Model description
 
@@ -43,22 +43,42 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
-- num_epochs: 10
+- num_epochs: 30
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 2.3454        | 1.0   | 158  | 1.2455          |
-| 2.1309        | 2.0   | 316  | 1.0947          |
-| 2.043         | 3.0   | 474  | 1.0464          |
-| 1.9615        | 4.0   | 632  | 1.0167          |
-| 1.8828        | 5.0   | 790  | 0.9873          |
-| 1.8103        | 6.0   | 948  | 0.9738          |
-| 1.7546        | 7.0   | 1106 | 0.9691          |
-| 1.7231        | 8.0   | 1264 | 0.9668          |
-| 1.7081        | 9.0   | 1422 | 0.9677          |
-| 1.6978        | 10.0  | 1580 | 0.9675          |
+| 2.3454        | 1.0   | 158  | 1.2439          |
+| 2.1288        | 2.0   | 316  | 1.0900          |
+| 2.0335        | 3.0   | 474  | 1.0394          |
+| 1.9315        | 4.0   | 632  | 0.9995          |
+| 1.804         | 5.0   | 790  | 0.9605          |
+| 1.6583        | 6.0   | 948  | 0.9411          |
+| 1.4994        | 7.0   | 1106 | 0.9283          |
+| 1.3388        | 8.0   | 1264 | 0.9158          |
+| 1.1894        | 9.0   | 1422 | 0.9103          |
+| 1.0616        | 10.0  | 1580 | 0.9027          |
+| 0.9461        | 11.0  | 1738 | 0.8963          |
+| 0.8447        | 12.0  | 1896 | 0.8922          |
+| 0.7575        | 13.0  | 2054 | 0.8887          |
+| 0.6817        | 14.0  | 2212 | 0.8803          |
+| 0.6192        | 15.0  | 2370 | 0.8761          |
+| 0.5669        | 16.0  | 2528 | 0.8715          |
+| 0.5196        | 17.0  | 2686 | 0.8719          |
+| 0.479         | 18.0  | 2844 | 0.8683          |
+| 0.4473        | 19.0  | 3002 | 0.8662          |
+| 0.4202        | 20.0  | 3160 | 0.8624          |
+| 0.397         | 21.0  | 3318 | 0.8590          |
+| 0.377         | 22.0  | 3476 | 0.8573          |
+| 0.3622        | 23.0  | 3634 | 0.8558          |
+| 0.3514        | 24.0  | 3792 | 0.8548          |
+| 0.3434        | 25.0  | 3950 | 0.8543          |
+| 0.3349        | 26.0  | 4108 | 0.8541          |
+| 0.332         | 27.0  | 4266 | 0.8538          |
+| 0.328         | 28.0  | 4424 | 0.8541          |
+| 0.3286        | 29.0  | 4582 | 0.8532          |
+| 0.3279        | 30.0  | 4740 | 0.8530          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,11 +20,11 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
+    "v_proj",
     "o_proj",
     "k_proj",
     "all-linear",
-    "v_proj"
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5df0c3aeedd7de88a9e5a7477d9a878b5c6d3ea1a97496be99b422a27bbc64f6
+oid sha256:2d78f98f66fae54ea02a26db118377f56bd46a4fad3e22a1c9380d770b22a02d
 size 218138576

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:56bb3002bb9eb9b76d5683f4c945445b2c56cf5f671b7bd03273380597f0f3bf
+oid sha256:1566104477e43dbd953449afb6f0974e2c7765e1042ea8d06d2a920964a1cde5
 size 5048