andreaskoepf
commited on
Commit
·
6fcad23
1
Parent(s):
011d716
Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,7 @@ library_name: transformers
|
|
24 |
# Open-Assistant Falcon 7B SFT OASST-TOP1 Model
|
25 |
|
26 |
This model is a fine-tuning of TII's [Falcon 7B](https://huggingface.co/tiiuae/falcon-7b) LLM.
|
27 |
-
It was trained with 11,123 top-1 (high-quality) demonstrations of the OASST data set (exported on
|
28 |
|
29 |
## Model Details
|
30 |
|
|
|
24 |
# Open-Assistant Falcon 7B SFT OASST-TOP1 Model
|
25 |
|
26 |
This model is a fine-tuning of TII's [Falcon 7B](https://huggingface.co/tiiuae/falcon-7b) LLM.
|
27 |
+
It was trained with 11,123 top-1 (high-quality) demonstrations of the OASST data set (exported on June 2, 2023) with a batch size of 128 for 8 epochs with LIMA style dropout (p=0.2) and a context-length of 2048 tokens.
|
28 |
|
29 |
## Model Details
|
30 |
|