Others quantization?

#1
by slavczays - opened

Any chance for models with lower quantization like q2_k (still usefull and better fit for mobile devices)?

Owner

Hi,
I uploaded 2 and 3bit quentizations for you :)

Thanks a lot bro!;)

Sign up or log in to comment