|
--- |
|
license: other |
|
license_name: microsoft-research-license |
|
tags: |
|
- merge |
|
- not-for-all-audiences |
|
--- |
|
# DarkForest 20B v2.0 - GGUF IMat quants |
|
|
|
[IMatrix file](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.fp16_512.imatrix?download=true) prepared witch context 4096 and 5521 chunks of wiki.test.raw. |
|
Original model info: [DarkForest-20B-v2.0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0) |
|
|
|
[q8_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0_Q8_0_imatrix.gguf?download=true): |
|
- [5521]8.5016, |
|
- Final estimate: PPL = 8.5016 +/- 0.02134 |
|
|
|
[q6_k](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q6_K_imatrix.gguf?download=true): |
|
- [5521]8.5046, |
|
- Final estimate: PPL = 8.5046 +/- 0.02136 |
|
|
|
[q5_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q5_0_imatrix.gguf?download=true): |
|
- [5521]8.4903, |
|
- Final estimate: PPL = 8.4903 +/- 0.02132 |
|
|
|
[q4_K_S](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_K_S_imatrix.gguf?download=true): |
|
- [5521]8.5880, |
|
- Final estimate: PPL = 8.5880 +/- 0.02162 |
|
|
|
[q4_K_M](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_K_M_imatrix.gguf?download=true): |
|
- [5521]8.5906, |
|
- Final estimate: PPL = 8.5906 +/- 0.02163 |
|
|
|
[q4_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_0_imatrix.gguf?download=true): |
|
- [5521]8.5610, |
|
- Final estimate: PPL = 8.5610 +/- 0.02151 |
|
|
|
[q3_K_M](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q3_K_M_imatrix.gguf?download=true): |
|
- [5521]8.7283, |
|
- Final estimate: PPL = 8.7283 +/- 0.02196 |
|
|
|
[q2_K](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q2_K_imatrix.gguf?download=true): |
|
- [5521]9.2445 |
|
- Final estimate: PPL = 9.2445 +/- 0.02351 |
|
|
|
[IQ2_XS](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ2_XS_imatrix.gguf?download=true): |
|
- [5521]9.8329, |
|
- Final estimate: PPL = 9.8329 +/- 0.02452 |
|
|
|
[IQ2_XSS](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ2_XXS_imatrix.gguf?download=true): |
|
- [5521]10.5170, |
|
- Final estimate: PPL = 10.5170 +/- 0.02651 |
|
|
|
[IQ1_S](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ1_S_imatrix.gguf?download=true): |
|
- [5521]13.9487, |
|
- Final estimate: PPL = 13.9487 +/- 0.03502 - UNUSABLE, model is too small to be coherent in 1 bit. |
|
|
|
|
|
All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel: |
|
<a href="https://www.buymeacoffee.com/TeeZee" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a> |
|
|
