abhinavkulkarni
/

VMware-open-llama-7b-open-instruct-w4-g128-awq

Text Generation

text-generation-inference

Model card Files Files and versions Community

Abhinav Kulkarni commited on Jul 20, 2023

Commit

d795ddd

•

1 Parent(s): 8b50c3d

Updated README

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -42,7 +42,7 @@ git clone https://github.com/abhinavkulkarni/llm-awq \
 ```python
 import torch
 from awq.quantize.quantizer import real_quantize_model_weight
-from transformers import AutoModelForCausalLM, AutoConfig, AutoTokenizer
 from accelerate import init_empty_weights, load_checkpoint_and_dispatch
 from huggingface_hub import snapshot_download

 ```python
 import torch
 from awq.quantize.quantizer import real_quantize_model_weight
+from transformers import AutoModelForCausalLM, AutoConfig, AutoTokenizer, TextStreamer
 from accelerate import init_empty_weights, load_checkpoint_and_dispatch
 from huggingface_hub import snapshot_download