Update README.md
Browse files
README.md
CHANGED
@@ -67,6 +67,13 @@ And after it, add:
|
|
67 |
[Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
|
68 |
|
69 |
3. Then save and close the file, and launch text-generation-webui as described below
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
70 |
|
71 |
## How to download and use this model in text-generation-webui
|
72 |
|
|
|
67 |
[Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
|
68 |
|
69 |
3. Then save and close the file, and launch text-generation-webui as described below
|
70 |
+
4. Install and compile AutoGPTQ from source - note that this requires compiling the CUDA kernel, which requires CUDA toolkit. This may be an issue for Windows users.
|
71 |
+
|
72 |
+
```
|
73 |
+
git clone https://github.com/PanQiWei/AutoGPTQ
|
74 |
+
cd AutoGPTQ
|
75 |
+
pip install . # This step requires CUDA toolkit installed
|
76 |
+
```
|
77 |
|
78 |
## How to download and use this model in text-generation-webui
|
79 |
|