Abhinav Kulkarni
commited on
Commit
•
d51df12
1
Parent(s):
8ca5887
Updated README
Browse files
README.md
CHANGED
@@ -23,7 +23,7 @@ Please refer to the AWQ quantization license ([link](https://github.com/llm-awq/
|
|
23 |
|
24 |
## CUDA Version
|
25 |
|
26 |
-
This model was successfully tested on CUDA driver v530.30.02 and runtime v11.7 with Python v3.10.11. Please note that AWQ requires NVIDIA GPUs with compute capability of
|
27 |
|
28 |
For Docker users, the `nvcr.io/nvidia/pytorch:23.06-py3` image is runtime v12.1 but otherwise the same as the configuration above and has also been verified to work.
|
29 |
|
@@ -84,7 +84,7 @@ output = model.generate(
|
|
84 |
repetition_penalty=1.1,
|
85 |
eos_token_id=tokenizer.eos_token_id
|
86 |
)
|
87 |
-
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
88 |
```
|
89 |
|
90 |
## Evaluation
|
|
|
23 |
|
24 |
## CUDA Version
|
25 |
|
26 |
+
This model was successfully tested on CUDA driver v530.30.02 and runtime v11.7 with Python v3.10.11. Please note that AWQ requires NVIDIA GPUs with compute capability of `8.0` or higher.
|
27 |
|
28 |
For Docker users, the `nvcr.io/nvidia/pytorch:23.06-py3` image is runtime v12.1 but otherwise the same as the configuration above and has also been verified to work.
|
29 |
|
|
|
84 |
repetition_penalty=1.1,
|
85 |
eos_token_id=tokenizer.eos_token_id
|
86 |
)
|
87 |
+
# print(tokenizer.decode(output[0], skip_special_tokens=True))
|
88 |
```
|
89 |
|
90 |
## Evaluation
|