42dot
/

42dot_LLM-PLM-1.3B

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

ykhwang commited on Sep 6, 2023

Commit

dc2109e

·

1 Parent(s): c2bf727

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -29,7 +29,7 @@ Pre-training took 9 days using 256 * NVIDIA A100 GPUs. Related settings are list
 | Params | Global batch size\* | Initial learning rate | Train iter.\* | Max length\* | Weight decay |
 | -- | -- | -- | -- | -- | -- |
-| 1.3B | 4.0M | 4E-4 | 1.0T | 8K | 0.1 |
 (\* unit: tokens)

 | Params | Global batch size\* | Initial learning rate | Train iter.\* | Max length\* | Weight decay |
 | -- | -- | -- | -- | -- | -- |
+| 1.3B | 4.0M | 4E-4 | 1.0T | 8,192 | 0.1 |
 (\* unit: tokens)