TheBloke

/

leo-hessianai-7B-AWQ

like 0

Hello,

i am running out of memory when loading the model with 12GB VRAM. What is the minimum memory needed for an AWQ quantized 7B model with group size of 128?

thx,
Manuel