Hugging Face's logo

imdatta0
/
mistralai_mistral_7b_v0.3_imdatta0_Magiccoder_evol_10k_reverse

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
mistralai_mistral_7b_v0.3_imdatta0_Magiccoder_evol_10k_reverse / README.md
imdatta0's picture
imdatta0
End of training
150df5a verified
preview code
|
raw
history blame contribute delete
3.49 kB
metadata
base_model: unsloth/mistral-7b-v0.3-bnb-4bit
library_name: peft
license: apache-2.0
tags:
  - unsloth
  - generated_from_trainer
model-index:
  - name: mistralai_mistral_7b_v0.3_imdatta0_Magiccoder_evol_10k_reverse
    results: []

mistralai_mistral_7b_v0.3_imdatta0_Magiccoder_evol_10k_reverse

This model is a fine-tuned version of unsloth/mistral-7b-v0.3-bnb-4bit on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.1504

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
1.1815 0.0262 4 1.2461
1.1779 0.0523 8 1.2277
1.2145 0.0785 12 1.2208
1.1589 0.1047 16 1.2399
1.2113 0.1308 20 1.2424
1.1171 0.1570 24 1.2347
1.2649 0.1832 28 1.2280
1.2005 0.2093 32 1.2154
1.1418 0.2355 36 1.2183
1.1896 0.2617 40 1.2063
1.2135 0.2878 44 1.2015
1.1641 0.3140 48 1.2015
1.1855 0.3401 52 1.2107
1.1493 0.3663 56 1.1929
1.168 0.3925 60 1.1938
1.2119 0.4186 64 1.2076
1.1207 0.4448 68 1.2077
1.1249 0.4710 72 1.1969
1.1242 0.4971 76 1.1923
1.2203 0.5233 80 1.1874
1.1168 0.5495 84 1.1766
1.1781 0.5756 88 1.1852
1.2153 0.6018 92 1.1785
1.213 0.6280 96 1.1682
1.1424 0.6541 100 1.1693
1.1577 0.6803 104 1.1702
1.1586 0.7065 108 1.1736
1.0325 0.7326 112 1.1546
1.1151 0.7588 116 1.1556
1.1153 0.7850 120 1.1539
1.1471 0.8111 124 1.1512
1.1408 0.8373 128 1.1488
1.1676 0.8635 132 1.1485
1.1049 0.8896 136 1.1489
1.1905 0.9158 140 1.1494
1.0539 0.9419 144 1.1500
1.0729 0.9681 148 1.1503
1.2164 0.9943 152 1.1504

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1