mjbuehler commited on
Commit
c91b69e
·
verified ·
1 Parent(s): 2c8b2a3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -7
README.md CHANGED
@@ -13,22 +13,25 @@ tags:
13
  - AI4Science
14
  - Materiomics
15
  - Biomateriomics
16
- base_model:
17
- - meta-llama/Meta-Llama-3.1-8B-Instruct
18
- - NousResearch/Hermes-3-Llama-3.1-8B
19
  datasets:
20
  - mlabonne/orpo-dpo-mix-40k
21
- - lamm-mit/magpie-ultra-v0.1-DPO
22
- - HuggingFaceH4/deita-10k-v0-sft
23
  - lamm-mit/bio-silk-mech-data-integrated
 
 
 
 
 
 
24
  ---
25
 
26
  # lamm-mit/Bioinspired-SmolLM-1.7B-Instruct
27
 
28
  This model was constructed from the SmolLM-1.7B base model using a combination of Continued Pre-training (CPT), Supervised fine-tuning (SFT), and DPO.
29
 
30
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/prfiePwzbYVqarvhnVYEt.png)
31
 
 
32
  The model was trained on a mix of publically available datasets and a corpus of around 5,000 scientific papers in the bio-inspired materials field. During the CPT phase, the raw text of all papers is used. During SFT and ORPO, the model is shown a high-quality mix of question-answer pairs and question-answer-rejected triples, respectively, along with other datasets to train the model for instructions and chat interactions.
33
 
34
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/2D3Jm0goTW_mvNRK2gKKU.png)
@@ -272,4 +275,4 @@ Please cite as:
272
  doi={10.1063/5.0203126},
273
  note={\url{https://doi.org/10.1063/5.0203126}}
274
  }
275
- ```
 
13
  - AI4Science
14
  - Materiomics
15
  - Biomateriomics
16
+ base_model:
17
+ - HuggingFaceTB/SmolLM-1.7B
 
18
  datasets:
19
  - mlabonne/orpo-dpo-mix-40k
 
 
20
  - lamm-mit/bio-silk-mech-data-integrated
21
+ - HuggingFaceTB/Magpie-Pro-300K-Filtered-H4
22
+ - HuggingFaceTB/self-oss-instruct-sc2-H4
23
+ - HuggingFaceTB/OpenHermes-2.5-H4
24
+ - HuggingFaceTB/everyday-conversations-llama3.1-2k
25
+ - HuggingFaceTB/instruct-data-basics-smollm-H4
26
+ license: apache-2.0
27
  ---
28
 
29
  # lamm-mit/Bioinspired-SmolLM-1.7B-Instruct
30
 
31
  This model was constructed from the SmolLM-1.7B base model using a combination of Continued Pre-training (CPT), Supervised fine-tuning (SFT), and DPO.
32
 
 
33
 
34
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/IhTyZRoOB11Qr9GRl54cx.png)
35
  The model was trained on a mix of publically available datasets and a corpus of around 5,000 scientific papers in the bio-inspired materials field. During the CPT phase, the raw text of all papers is used. During SFT and ORPO, the model is shown a high-quality mix of question-answer pairs and question-answer-rejected triples, respectively, along with other datasets to train the model for instructions and chat interactions.
36
 
37
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/2D3Jm0goTW_mvNRK2gKKU.png)
 
275
  doi={10.1063/5.0203126},
276
  note={\url{https://doi.org/10.1063/5.0203126}}
277
  }
278
+ ```