Update README.md
Browse files
README.md
CHANGED
|
@@ -30,8 +30,7 @@ More information needed
|
|
| 30 |
|
| 31 |
## Training and evaluation data
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
## Training procedure
|
| 36 |
|
| 37 |
### Training hyperparameters
|
|
|
|
| 30 |
|
| 31 |
## Training and evaluation data
|
| 32 |
|
| 33 |
+
- this was trained in `bf16`. the [official recommendation](https://github.com/facebookresearch/mega#tips) is fp32 - still exploring this.
|
|
|
|
| 34 |
## Training procedure
|
| 35 |
|
| 36 |
### Training hyperparameters
|