abhinavkulkarni
/

mosaicml-mpt-30b-chat-w4-g128-awq

Text Generation

text-generation-inference

Model card Files Files and versions Community

abhinavkulkarni commited on Sep 7, 2023

Commit

6cebd72

•

1 Parent(s): b04d760

Upload config

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -45,7 +45,7 @@
   "norm_type": "low_precision_layernorm",
   "resid_pdrop": 0,
   "tokenizer_name": "sam-mosaic/gpt-neox-20b-chatml",
-  "torch_dtype": "bfloat16",
   "transformers_version": "4.33.1",
   "use_cache": false,
   "verbose": 0,

   "norm_type": "low_precision_layernorm",
   "resid_pdrop": 0,
   "tokenizer_name": "sam-mosaic/gpt-neox-20b-chatml",
+  "torch_dtype": "float16",
   "transformers_version": "4.33.1",
   "use_cache": false,
   "verbose": 0,