shouray/Condition-Model-3

README.md

@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [TheBloke/Llama-2-13B-chat-GPTQ](https://huggingface.co/TheBloke/Llama-2-13B-chat-GPTQ) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0570
+- Loss: 0.1030
 
 ## Model description
 
@@ -44,33 +44,21 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 2
-- training_steps: 30
+- training_steps: 100
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 0.8           | 1.0   | 1    | 1.0124          |
-| 0.3868        | 2.0   | 3    | 0.8254          |
-| 0.3112        | 3.0   | 5    | 0.5926          |
-| 0.4737        | 4.0   | 6    | 0.5064          |
-| 0.4067        | 5.0   | 7    | 0.4373          |
-| 0.163         | 6.0   | 9    | 0.3445          |
-| 0.1325        | 7.0   | 11   | 0.2647          |
-| 0.2128        | 8.0   | 12   | 0.2263          |
-| 0.1826        | 9.0   | 13   | 0.1899          |
-| 0.0706        | 10.0  | 15   | 0.1438          |
-| 0.0574        | 11.0  | 17   | 0.1187          |
-| 0.0971        | 12.0  | 18   | 0.1078          |
-| 0.0864        | 13.0  | 19   | 0.0992          |
-| 0.0372        | 14.0  | 21   | 0.0841          |
-| 0.0318        | 15.0  | 23   | 0.0729          |
-| 0.0572        | 16.0  | 24   | 0.0688          |
-| 0.0539        | 17.0  | 25   | 0.0657          |
-| 0.0249        | 18.0  | 27   | 0.0609          |
-| 0.0231        | 19.0  | 29   | 0.0577          |
-| 0.044         | 20.0  | 30   | 0.0570          |
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 1.0135        | 0.9412 | 12   | 0.4043          |
+| 0.145         | 1.9608 | 25   | 0.1342          |
+| 0.0703        | 2.9804 | 38   | 0.1080          |
+| 0.0531        | 4.0    | 51   | 0.1023          |
+| 0.0532        | 4.9412 | 63   | 0.1040          |
+| 0.0476        | 5.9608 | 76   | 0.1037          |
+| 0.0459        | 6.9804 | 89   | 0.1028          |
+| 0.0449        | 7.8431 | 100  | 0.1030          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,10 +20,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "o_proj",
     "k_proj",
-    "v_proj",
-    "q_proj"
+    "o_proj",
+    "q_proj",
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a790be2ccc118bf5f53205683a05aaab2038957693175722da253a20b2dc31d0
+oid sha256:a9ae492e16ddbddbaa034401c54f7a806e99118b05b17b2a1b2326058b708c77
 size 52471504

runs/Jun26_13-25-12_13d3a7536ccc/events.out.tfevents.1719408313.13d3a7536ccc.162.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:265ba577d736b5dc902d131937d124e3361cf399368842883b7035dd41b99ad0
+size 9696

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5cec7146a65f7d55f647021da9410ca0431bcca38582b3cd4a842a4c6032570b
+oid sha256:1b381e15e59d6da5795cafb71e3b8c1b0aec8cb6221559305809553e459ae785
 size 5112