Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
GPTQ group size effects vRAM usage
#18
by
sgjohnson
- opened
Unless this isn't the case, many of the models claim GPTQ group size effects vRAM usage, but you can only select precision here. Is group size covered by "Auto"?
Auto covers no_split modules, which generally should group those together. If you have an example model I can run a test to confirm this :)