InferenceIllusionist
commited on
Commit
•
50307b6
1
Parent(s):
8f1ecff
Update README.md
Browse files
README.md
CHANGED
@@ -8,14 +8,19 @@ license: cc-by-nc-4.0
|
|
8 |
- Model creator: [Sao10K](https://huggingface.co/Sao10K/)
|
9 |
- Original model: [Fimbulvetr-11B-v2](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)
|
10 |
|
11 |
-
|
12 |
|
13 |
-
|
|
|
14 |
|
15 |
Quantized from fp16 with love. Importance matrix file [Fimbulvetr-11B-v2-imatrix.dat](https://huggingface.co/InferenceIllusionist/Fimbulvetr-11B-v2-iMat-GGUF/blob/main/Fimbulvetr-11B-v2-imatrix.dat) was calculated using Q8_0.
|
16 |
|
17 |
-
|
18 |
-
|
|
|
|
|
|
|
|
|
19 |
|
20 |
![Fox1](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2/resolve/main/cute1.jpg)
|
21 |
|
|
|
8 |
- Model creator: [Sao10K](https://huggingface.co/Sao10K/)
|
9 |
- Original model: [Fimbulvetr-11B-v2](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)
|
10 |
|
11 |
+
<i>Looking for Q3/Q4/Q5 quants? See the link in the model card below.</i>
|
12 |
|
13 |
+
|
14 |
+
All credits to Sao10K for the original model. This is just a quick test of the new quantization types such as IQ_3S in an attempt to further reduce VRAM requirements.
|
15 |
|
16 |
Quantized from fp16 with love. Importance matrix file [Fimbulvetr-11B-v2-imatrix.dat](https://huggingface.co/InferenceIllusionist/Fimbulvetr-11B-v2-iMat-GGUF/blob/main/Fimbulvetr-11B-v2-imatrix.dat) was calculated using Q8_0.
|
17 |
|
18 |
+
<b>Please Note: </b> Inferencing for newer formats (IQ3_S, IQ4_NL, etc) was tested on llama.cpp. These newer quants may not work with other methods yet (as of 2/25/24).
|
19 |
+
|
20 |
+
Original model card details below.
|
21 |
+
|
22 |
+
---
|
23 |
+
|
24 |
|
25 |
![Fox1](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2/resolve/main/cute1.jpg)
|
26 |
|