is it possible to make this work with huggingface/text-generation-inference ?
#5
by
captainst
- opened
I tried the model with text-generation-inference. The docker run reports "weight gptq_bits does not exist" error.
I am wondering if it is possible to make this model work with the huggingface text-generation-inference. That'll be terrific.
Thanks, I am interested in the same. It would be then useful to use this as a code copilot using HF VScode extension.
@captainst : Have you managed to figure this out?