EXL2 quants of Qwen2-VL-72B-Instruct
4.00 bits per weight
4.50 bits per weight
5.00 bits per weight
6.00 bits per weight
(2.3bpw to 3.5bpw revisions are in also this repo, but they are unstable. Working on it.)
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.