Downtown-Case/Qwen_Qwen2.5-32B-Base-exl2-3.92bpw
Text Generation
•
Updated
•
12
•
1
24GB VRAM Optimal Quants
When asked what I use locally on a 24GB card, this is what I point to. I favor exl2s for long context, GGUF for very short context.
-
-
Downtown-Case/Qwen_Qwen2.5-32B-Base-exl2-3.62bpw
Text Generation • Updated • 9 • 1 -
Downtown-Case/Qwen_Qwen2.5-32B-Base-exl2-3.75bpw
Text Generation • Updated • 11 • 1 -
Downtown-Case/Star-Command-R-Lite-32B-v1-exl2-4bpw
Text Generation • Updated • 25 • 1 -
Downtown-Case/aws-prototyping_MegaBeam-Mistral-7B-512K-exl2-8.0bpw
Updated • 13 • 1 -
ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16
Text Generation • Updated • 732 • 43 -
async0x42/c4ai-command-r-08-2024-exl2_4bpw
Text Generation • Updated • 30 • 1 -
Apel-sin/codestral-22B-v0.1-exl2
Updated • 2 -
Downtown-Case/Qwen_Qwen2.5-32B-Base-exl2-4.1bpw
Text Generation • Updated • 11 -
legraphista/Qwen2.5-32B-Instruct-IMat-GGUF
Text Generation • Updated • 937 -
Downtown-Case/EVA-UNIT-01_EVA-Qwen2.5-32B-v0.1-exl2-4.1bpw
Text Generation • Updated • 7