TheBloke jlzhou commited on
Commit
bbedaae
1 Parent(s): d44b281

fix: quantize param in TGI example (#8)

Browse files

- fix: quantize param in TGI example (f6e48e7b4c03f51cfb4335c550c7a53cfd02e380)


Co-authored-by: Junlin Zhou <[email protected]>

Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -208,7 +208,7 @@ It's recommended to use TGI version 1.1.0 or later. The official Docker containe
208
  Example Docker parameters:
209
 
210
  ```shell
211
- --model-id TheBloke/Mistral-7B-OpenOrca-GPTQ --port 3000 --quantize awq --max-input-length 3696 --max-total-tokens 4096 --max-batch-prefill-tokens 4096
212
  ```
213
 
214
  Example Python code for interfacing with TGI (requires huggingface-hub 0.17.0 or later):
 
208
  Example Docker parameters:
209
 
210
  ```shell
211
+ --model-id TheBloke/Mistral-7B-OpenOrca-GPTQ --port 3000 --quantize gptq --max-input-length 3696 --max-total-tokens 4096 --max-batch-prefill-tokens 4096
212
  ```
213
 
214
  Example Python code for interfacing with TGI (requires huggingface-hub 0.17.0 or later):