Update README.md
Browse files
README.md
CHANGED
|
@@ -8,11 +8,17 @@ tags:
|
|
| 8 |
|
| 9 |
# redpajama gptq
|
| 10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
|
| 13 |
## Usage
|
| 14 |
|
| 15 |
|
|
|
|
| 16 |
install auto-GPTQ
|
| 17 |
|
| 18 |
```bash
|
|
|
|
| 8 |
|
| 9 |
# redpajama gptq
|
| 10 |
|
| 11 |
+
<a href="https://colab.research.google.com/gist/pszemraj/86d2e8485df182302646ed2c5a637059/inference-with-redpajama-incite-chat-3b-v1-gptq-4bit-128g.ipynb">
|
| 12 |
+
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
|
| 13 |
+
</a>
|
| 14 |
+
|
| 15 |
+
A GPTQ quantization of the [RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) via auto-gptq.
|
| 16 |
|
| 17 |
|
| 18 |
## Usage
|
| 19 |
|
| 20 |
|
| 21 |
+
> Note that you cannot load directly from the hub with `auto_gptq` yet - if needed you can use [this function](https://gist.github.com/pszemraj/8368cba3400bda6879e521a55d2346d0) to download using the repo name.
|
| 22 |
install auto-GPTQ
|
| 23 |
|
| 24 |
```bash
|