Update README.md
Browse files
README.md
CHANGED
@@ -12,6 +12,10 @@ This repo contains an experimantal GPTQ 4bit model for [Falcon-7B-Instruct](http
|
|
12 |
|
13 |
It is the result of quantising to 4bit using [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ).
|
14 |
|
|
|
|
|
|
|
|
|
15 |
## EXPERIMENTAL
|
16 |
|
17 |
Please note this is an experimental first model. Support for it is currently quite limited.
|
@@ -29,7 +33,7 @@ There is also provisional AutoGPTQ support in text-generation-webui.
|
|
29 |
|
30 |
However at the time I'm writing this, a commit is needed to text-generation-webui to enable it to load this model.
|
31 |
|
32 |
-
|
33 |
|
34 |
To get it working before the PR is merged, you will need to:
|
35 |
1. Edit `text-generation-webui/modules/AutoGPTQ_loader.py`
|
@@ -47,7 +51,13 @@ And after it, add:
|
|
47 |
|
48 |
[Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
|
49 |
|
50 |
-
3.
|
|
|
|
|
|
|
|
|
|
|
|
|
51 |
|
52 |
## How to download and use this model in text-generation-webui
|
53 |
|
|
|
12 |
|
13 |
It is the result of quantising to 4bit using [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ).
|
14 |
|
15 |
+
## Need support? Want to discuss? I now have a Discord!
|
16 |
+
|
17 |
+
Join me at: https://discord.gg/UBgz4VXf
|
18 |
+
|
19 |
## EXPERIMENTAL
|
20 |
|
21 |
Please note this is an experimental first model. Support for it is currently quite limited.
|
|
|
33 |
|
34 |
However at the time I'm writing this, a commit is needed to text-generation-webui to enable it to load this model.
|
35 |
|
36 |
+
A [PR has been opened](https://github.com/oobabooga/text-generation-webui/pull/2367) which will provide support for this model.
|
37 |
|
38 |
To get it working before the PR is merged, you will need to:
|
39 |
1. Edit `text-generation-webui/modules/AutoGPTQ_loader.py`
|
|
|
51 |
|
52 |
[Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
|
53 |
|
54 |
+
3. Install `einops` if you don't already have it:
|
55 |
+
|
56 |
+
```
|
57 |
+
pip install einops
|
58 |
+
```
|
59 |
+
|
60 |
+
4. Then launch text-generation-webui as described below
|
61 |
|
62 |
## How to download and use this model in text-generation-webui
|
63 |
|