Any chance of an AWQ version of the 405B base model?
#12
by
lodrick-the-lafted
- opened
There's not exactly a surplus of large base models to try, I would really love to see an AWQ quant of L3.1-405B base.
There's not exactly a surplus of large base models to try, I would really love to see an AWQ quant of L3.1-405B base.
Hi here @lodrick-the-lafted , sure, we do have the base quant private, I'll try to add the model card and make it public later today or over the weekend and ping you back, thanks 🤗
It's been a while, maybe take another look?