Update README
Browse files
README.md
CHANGED
|
@@ -1,3 +1,30 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
---
|
| 6 |
+
## Information
|
| 7 |
+
|
| 8 |
+
This is a Exl2 quantized version of [airoboros-mistral2.2-7b](https://huggingface.co/teknium/airoboros-mistral2.2-7b)
|
| 9 |
+
|
| 10 |
+
Please refer to the original creator for more information.
|
| 11 |
+
|
| 12 |
+
Calibration dataset: [wikitext](https://huggingface.co/datasets/wikitext/tree/refs%2Fconvert%2Fparquet/wikitext-2-v1/test)
|
| 13 |
+
|
| 14 |
+
## Branches:
|
| 15 |
+
|
| 16 |
+
- main: Measurement files
|
| 17 |
+
- 4bpw: 4 bits per weight
|
| 18 |
+
- 5bpw: 5 bits per weight
|
| 19 |
+
- 6bpw: 6 bits per weight
|
| 20 |
+
|
| 21 |
+
## Notes
|
| 22 |
+
|
| 23 |
+
- 6bpw is recommended for the best quality to vram usage ratio (assuming you have enough vram).
|
| 24 |
+
- Please ask for more bpws in the community tab if necessary.
|
| 25 |
+
|
| 26 |
+
## Donate?
|
| 27 |
+
|
| 28 |
+
All my infrastructure and cloud expenses are paid out of pocket. If you'd like to donate, you can do so here: https://ko-fi.com/kingbri
|
| 29 |
+
|
| 30 |
+
You should not feel obligated to donate, but if you do, I'd appreciate it.
|