VRAM require
#1
by
paperplanedeemo
- opened
exm. Could you please tell me how much VRAM does this model require?I haven't tested it yet, so I want to choose a configuration that meets the requirements for deployment. Thank you!
Not sure, but I was able to get to around 190K context with about 110 GB VRAM across my 5x 3090s using the exl2 quant. 1M is going to be some ungodly number.
Not sure, but I was able to get to around 190K context with about 110 GB VRAM across my 5x 3090s using the exl2 quant. 1M is going to be some ungodly number.
thank you so much