Llama 3.1 GPTQ, AWQ, and BNB Quants - a hugging-quants Collection

hugging-quants 's Collections

Gemma2 AWQ Quants

Llama 3.2 3B & 1B GGUF Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

updated Sep 26

Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • Updated Sep 13 • 3.98k • 37
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4

Text Generation • Updated Sep 16 • 514 • 5
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 17.4k • 16
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 222k • 87
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit

Text Generation • Updated Sep 3 • 18.5k • 22
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 48.5k • 17
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 393k • 60
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4

Text Generation • Updated Aug 8 • 3.13k • 7
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 52.9k • 19