2.7bpw?
Hey! Thank you so much for your quants. Was just wondering if there's a chance for a 2.7bpw for this model?
Hey! Thank you so much for your quants. Was just wondering if there's a chance for a 2.7bpw for this model?
Hi! Sorry, I have been out for a few days. Are you still interested in a 2.7? I could get one cooking for you.
No worries at all! And yes, if it's OK with you. Thank you!!
No worries at all! And yes, if it's OK with you. Thank you!!
It looks like maybe you saw it already but here you go!.
https://huggingface.co/BigHuggyD/FluffyKaeloky_Luminum-v0.1-123B_exl2_2.7bpw_h6
Apologies if you download already. I realized after uploading that I had modified the max embeddings for testing and never changed them back.
You can either download the corrected config.json, or edit your copy to put back max embeddings to 131072
Thanks for the heads up and thank you very much for the quant!