mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-4bit_g64-HQQ
Text Generation
•
Updated
•
35
•
9
Mixtral HQQ Quantized Models
4-bit and 2-bit Mixtral models quantized using https://github.com/mobiusml/hqq
-
-
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-2bit_g16_s128-HQQ
Text Generation • Updated • 34 • 9 -
mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-2bit_g16_s128-HQQ
Text Generation • Updated • 41 • 4 -
mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-4bit_g64-HQQ
Text Generation • Updated • 20 • 1
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-HQQ
Text Generation • Updated • 50 • 38Note If you are considering 2-bit instruct model use this one.
mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-attn-4bit-moe-2bit-HQQ
Text Generation • Updated • 35 • 6Note If you are considering 2-bit base model use this one.
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-metaoffload-HQQ
Text Generation • Updated • 62 • 15Note If you are considering 2-bit base model but is GPU pure this is a good option. Requires 13GB of RAM, but it will be slower.
-
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-3bit-metaoffload-HQQ
Text Generation • Updated • 28 • 13 -
mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bitgs8-metaoffload-HQQ
Text Generation • Updated • 40 • 20