File size: 702 Bytes
90ea747 706a7c2 1702351 73d8976 3a6d157 1702351 74f166b 1702351 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
---
datasets:
- osunlp/SMolInstruct
language:
- en
base_model:
- mistralai/Mistral-7B-Instruct-v0.3
tags:
- chemistry
library_name: mlx
---
Finetuned on SmolInstruct's property prediction instruction dataset and HoneyBee's instruction dataset.
[LoRA Config Parameters]
train: true,
fine_tune_type: lora,
seed: 0,
num_layers: 8,
batch_size: 2,
iters: 1000,
val_batches: 25,
learning_rate: 1e-5,
steps_per_report: 10,
steps_per_eval: 200,
resume_adapter_file: null,
adapter_path: "adapters",
save_every: 100,
test: false,
test_batches: 100,
max_seq_length: 2048,
grad_checkpoint: false,
lora_parameters:
keys: ["self_attn.q_proj", "self_attn.v_proj"]
rank: 8
alpha: 8
dropout: 0.0
scale: 20.0 |