Inference
#1
by
bartowski
- opened
This can't be run with autogptq or exllama right?
Hey @bartowski ,
No, you need to install awq package as detailed in the model card just like for GPTQ you need to install auto_gptq or exllama. Otherwise it's pretty similar to how it works.
abhinavkulkarni
changed discussion status to
closed