HIGGS
Collection
Models prequantized with [HIGGS](https://arxiv.org/abs/2411.17525) zero-shot quantization. Requires the latest `transformers` to run.
•
17 items
•
Updated
•
5
Groups | Version | Filter | n-shot | Metric | Value | Stderr | ||
---|---|---|---|---|---|---|---|---|
mmlu | N/A | none | 0 | acc | ↑ | 0.8202 | ± | 0.0031 |
- humanities | N/A | none | 5 | acc | ↑ | 0.8013 | ± | 0.0056 |
- other | N/A | none | 5 | acc | ↑ | 0.8455 | ± | 0.0062 |
- social_sciences | N/A | none | 5 | acc | ↑ | 0.8898 | ± | 0.0055 |
- stem | N/A | none | 5 | acc | ↑ | 0.7555 | ± | 0.0073 |