Update README.md
Browse files
README.md
CHANGED
@@ -16,6 +16,7 @@ pip install bitblas #to use the bitblas backend
|
|
16 |
```
|
17 |
|
18 |
Then you can use the sample code below:
|
|
|
19 |
``` Python
|
20 |
import torch
|
21 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
@@ -72,6 +73,7 @@ if(backend == 'gemlite'):
|
|
72 |
gemlite.core.GemLiteLinear.cache_config('/tmp/gemlite_config.json')
|
73 |
```
|
74 |
|
|
|
75 |
Run with <a href="https://github.com/vllm-project/vllm/">vllm</a>:
|
76 |
```Python
|
77 |
##################################################################
|
|
|
16 |
```
|
17 |
|
18 |
Then you can use the sample code below:
|
19 |
+
## Transformers 🤗
|
20 |
``` Python
|
21 |
import torch
|
22 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
73 |
gemlite.core.GemLiteLinear.cache_config('/tmp/gemlite_config.json')
|
74 |
```
|
75 |
|
76 |
+
## VLLM
|
77 |
Run with <a href="https://github.com/vllm-project/vllm/">vllm</a>:
|
78 |
```Python
|
79 |
##################################################################
|