reshinthadith
commited on
Commit
·
5d3723b
1
Parent(s):
fd8d746
Update README.md
Browse files
README.md
CHANGED
@@ -55,7 +55,7 @@ print(tokenizer.decode(tokens[0], skip_special_tokens=True))
|
|
55 |
|
56 |
### Training Procedure
|
57 |
|
58 |
-
Models are pre-trained on the aforementioned dataset in mixed-precision (FP16), optimized with Adam, and trained using the NeoX tokenizer with a vocabulary size of 50,257. We outline the complete hyperparameters choices in the project's GitHub repository
|
59 |
|
60 |
## Use and Limitations
|
61 |
|
@@ -65,7 +65,7 @@ These models are intended to be used by all individuals as foundational models f
|
|
65 |
|
66 |
### Limitations and bias
|
67 |
|
68 |
-
The pre-training dataset may
|
69 |
|
70 |
## Citations
|
71 |
|
|
|
55 |
|
56 |
### Training Procedure
|
57 |
|
58 |
+
Models are pre-trained on the aforementioned dataset in mixed-precision (FP16), optimized with Adam, and trained using the NeoX tokenizer with a vocabulary size of 50,257. We outline the complete hyperparameters choices in the project's [GitHub repository](https://github.com/Stability-AI/StableLM-staging/blob/main/configs/stablelm-base-alpha-7b.yaml).
|
59 |
|
60 |
## Use and Limitations
|
61 |
|
|
|
65 |
|
66 |
### Limitations and bias
|
67 |
|
68 |
+
The pre-training dataset may have contained offensive or inappropriate content even after applying data cleansing filters which can be reflected in the model generated text. We recommend users exercise reasonable caution when using these models in production systems. Do not use the models for any applications that may cause harm or distress to individuals or groups.
|
69 |
|
70 |
## Citations
|
71 |
|