That's awesome! But...
#1
by
Anonimus12345678902
- opened
That's awesome! But man, I wish I could download Theia-21B-v1-Q4_K_S.gguf to run on my 12 gig 3060 video card. Theia-21B-v1-Q3_K_M_M.gguf is too dumb and Theia-21B-v1-Q4_K_M_M.gguf is too slow. I think many people who have such a video card will agree with me.
Maybe try imatrix quants ? i1-Q3_K_L maybe.