File size: 2,988 Bytes
8327e45 b7b5fc0 8327e45 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
license: other
license_name: microsoft-research-license
tags:
- merge
- not-for-all-audiences
---
# DarkForest 20B v2.0 - GGUF IMat quants
[IMatrix file](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.fp16_512.imatrix?download=true) prepared witch context 4096 and 5521 chunks of wiki.test.raw.
Original model info: [DarkForest-20B-v2.0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0)
[q8_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0_Q8_0_imatrix.gguf?download=true):
- [5521]8.5016,
- Final estimate: PPL = 8.5016 +/- 0.02134
[q6_k](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q6_K_imatrix.gguf?download=true):
- [5521]8.5046,
- Final estimate: PPL = 8.5046 +/- 0.02136
[q5_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q5_0_imatrix.gguf?download=true):
- [5521]8.4903,
- Final estimate: PPL = 8.4903 +/- 0.02132
[q4_K_S](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_K_S_imatrix.gguf?download=true):
- [5521]8.5880,
- Final estimate: PPL = 8.5880 +/- 0.02162
[q4_K_M](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_K_M_imatrix.gguf?download=true):
- [5521]8.5906,
- Final estimate: PPL = 8.5906 +/- 0.02163
[q4_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_0_imatrix.gguf?download=true):
- [5521]8.5610,
- Final estimate: PPL = 8.5610 +/- 0.02151
[q3_K_M](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q3_K_M_imatrix.gguf?download=true):
- [5521]8.7283,
- Final estimate: PPL = 8.7283 +/- 0.02196
[q2_K](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q2_K_imatrix.gguf?download=true):
- [5521]9.2445
- Final estimate: PPL = 9.2445 +/- 0.02351
[IQ2_XS](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ2_XS_imatrix.gguf?download=true):
- [5521]9.8329,
- Final estimate: PPL = 9.8329 +/- 0.02452
[IQ2_XSS](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ2_XXS_imatrix.gguf?download=true):
- [5521]10.5170,
- Final estimate: PPL = 10.5170 +/- 0.02651
[IQ1_S](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ1_S_imatrix.gguf?download=true):
- [5521]13.9487,
- Final estimate: PPL = 13.9487 +/- 0.03502 - UNUSABLE, model is too small to be coherent in 1 bit.
All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel:
<a href="https://www.buymeacoffee.com/TeeZee" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a>
|