ISTA-DASLab
/

Qwen2-72B-Instruct-AQLM-PV-1bit-1x16

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

No model card

New: Create and edit this model card directly on the website!

Contribute a Model Card

Downloads last month: 73

Safetensors

Model size

7.48B params

Tensor type

FP16

·

I16

·

Inference Examples

Text Generation

Inference API (serverless) is not available, repository is disabled.

Collection including ISTA-DASLab/Qwen2-72B-Instruct-AQLM-PV-1bit-1x16

AQLM+PV

Official AQLM quantizations for "PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression": https://arxiv.org/abs/2405.14852 • 21 items • Updated 1 day ago • 15