Update README.md
Browse files
README.md
CHANGED
@@ -13,22 +13,25 @@ tags:
|
|
13 |
- AI4Science
|
14 |
- Materiomics
|
15 |
- Biomateriomics
|
16 |
-
base_model:
|
17 |
-
-
|
18 |
-
- NousResearch/Hermes-3-Llama-3.1-8B
|
19 |
datasets:
|
20 |
- mlabonne/orpo-dpo-mix-40k
|
21 |
-
- lamm-mit/magpie-ultra-v0.1-DPO
|
22 |
-
- HuggingFaceH4/deita-10k-v0-sft
|
23 |
- lamm-mit/bio-silk-mech-data-integrated
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
---
|
25 |
|
26 |
# lamm-mit/Bioinspired-SmolLM-1.7B-Instruct
|
27 |
|
28 |
This model was constructed from the SmolLM-1.7B base model using a combination of Continued Pre-training (CPT), Supervised fine-tuning (SFT), and DPO.
|
29 |
|
30 |
-
![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/prfiePwzbYVqarvhnVYEt.png)
|
31 |
|
|
|
32 |
The model was trained on a mix of publically available datasets and a corpus of around 5,000 scientific papers in the bio-inspired materials field. During the CPT phase, the raw text of all papers is used. During SFT and ORPO, the model is shown a high-quality mix of question-answer pairs and question-answer-rejected triples, respectively, along with other datasets to train the model for instructions and chat interactions.
|
33 |
|
34 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/2D3Jm0goTW_mvNRK2gKKU.png)
|
@@ -272,4 +275,4 @@ Please cite as:
|
|
272 |
doi={10.1063/5.0203126},
|
273 |
note={\url{https://doi.org/10.1063/5.0203126}}
|
274 |
}
|
275 |
-
```
|
|
|
13 |
- AI4Science
|
14 |
- Materiomics
|
15 |
- Biomateriomics
|
16 |
+
base_model:
|
17 |
+
- HuggingFaceTB/SmolLM-1.7B
|
|
|
18 |
datasets:
|
19 |
- mlabonne/orpo-dpo-mix-40k
|
|
|
|
|
20 |
- lamm-mit/bio-silk-mech-data-integrated
|
21 |
+
- HuggingFaceTB/Magpie-Pro-300K-Filtered-H4
|
22 |
+
- HuggingFaceTB/self-oss-instruct-sc2-H4
|
23 |
+
- HuggingFaceTB/OpenHermes-2.5-H4
|
24 |
+
- HuggingFaceTB/everyday-conversations-llama3.1-2k
|
25 |
+
- HuggingFaceTB/instruct-data-basics-smollm-H4
|
26 |
+
license: apache-2.0
|
27 |
---
|
28 |
|
29 |
# lamm-mit/Bioinspired-SmolLM-1.7B-Instruct
|
30 |
|
31 |
This model was constructed from the SmolLM-1.7B base model using a combination of Continued Pre-training (CPT), Supervised fine-tuning (SFT), and DPO.
|
32 |
|
|
|
33 |
|
34 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/IhTyZRoOB11Qr9GRl54cx.png)
|
35 |
The model was trained on a mix of publically available datasets and a corpus of around 5,000 scientific papers in the bio-inspired materials field. During the CPT phase, the raw text of all papers is used. During SFT and ORPO, the model is shown a high-quality mix of question-answer pairs and question-answer-rejected triples, respectively, along with other datasets to train the model for instructions and chat interactions.
|
36 |
|
37 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/2D3Jm0goTW_mvNRK2gKKU.png)
|
|
|
275 |
doi={10.1063/5.0203126},
|
276 |
note={\url{https://doi.org/10.1063/5.0203126}}
|
277 |
}
|
278 |
+
```
|