Update README.md
Browse files
README.md
CHANGED
@@ -121,15 +121,14 @@ print(tokenizer.decode(output[0]))
|
|
121 |
|
122 |
**gptq_model-4bit--1g.safetensors**
|
123 |
|
124 |
-
This will work with AutoGPTQ
|
125 |
|
126 |
It was created without groupsize to reduce VRAM requirements, and with `desc_act` (act-order) to improve inference quality.
|
127 |
|
128 |
* `gptq_model-4bit--1g.safetensors`
|
129 |
-
* Works
|
130 |
* At this time it does not work with AutoGPTQ Triton, but support will hopefully be added in time.
|
131 |
-
* Works with text-generation-webui using `--
|
132 |
-
* At this time it does NOT work with one-click-installers
|
133 |
* Does not work with any version of GPTQ-for-LLaMa
|
134 |
* Parameters: Groupsize = None. Act order (desc_act)
|
135 |
|
|
|
121 |
|
122 |
**gptq_model-4bit--1g.safetensors**
|
123 |
|
124 |
+
This will work with AutoGPTQ 0.2.0 and later.
|
125 |
|
126 |
It was created without groupsize to reduce VRAM requirements, and with `desc_act` (act-order) to improve inference quality.
|
127 |
|
128 |
* `gptq_model-4bit--1g.safetensors`
|
129 |
+
* Works AutoGPTQ 0.2.0 and later.
|
130 |
* At this time it does not work with AutoGPTQ Triton, but support will hopefully be added in time.
|
131 |
+
* Works with text-generation-webui using `--trust-remote-code`
|
|
|
132 |
* Does not work with any version of GPTQ-for-LLaMa
|
133 |
* Parameters: Groupsize = None. Act order (desc_act)
|
134 |
|