InferenceIllusionist
commited on
Commit
•
5ae74ce
1
Parent(s):
69a30ac
Update README.md
Browse files
README.md
CHANGED
@@ -4,20 +4,20 @@ tags:
|
|
4 |
license: cc-by-nc-4.0
|
5 |
---
|
6 |
|
7 |
-
|
8 |
-
- Model creator: [Sao10K](https://huggingface.co/Sao10K/)
|
9 |
-
- Original model: [Fimbulvetr-11B-v2](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)
|
10 |
|
11 |
-
|
|
|
|
|
|
|
12 |
|
13 |
|
14 |
All credits to Sao10K for the original model. This is just a quick test of the new quantization types such as IQ_3S in an attempt to further reduce VRAM requirements.
|
15 |
|
16 |
Quantized from fp16 with love. Importance matrix file [Fimbulvetr-11B-v2-imatrix.dat](https://huggingface.co/InferenceIllusionist/Fimbulvetr-11B-v2-iMat-GGUF/blob/main/Fimbulvetr-11B-v2-imatrix.dat) was calculated using Q8_0.
|
17 |
|
18 |
-
<b>Please Note: </b> Inferencing for newer formats (IQ3_S, IQ4_NL, etc) was tested on llama.cpp. These newer quants may not work with other methods yet (as of 2/25/24).
|
19 |
|
20 |
-
|
21 |
|
22 |
---
|
23 |
|
@@ -28,7 +28,7 @@ Original model card details below.
|
|
28 |
|
29 |
**https://huggingface.co/Sao10K/Fimbulvetr-11B-v2-GGUF <------ GGUF**
|
30 |
|
31 |
-
Fimbulvetr-v2 - A Solar-Based Model
|
32 |
|
33 |
Prompt Formats - Alpaca or Vicuna. Either one works fine.
|
34 |
Recommended SillyTavern Presets - Universal Light
|
|
|
4 |
license: cc-by-nc-4.0
|
5 |
---
|
6 |
|
7 |
+
<h3> Model Card for Fimbulvetr-11B-v2-iMat-GGUF</h3>
|
|
|
|
|
8 |
|
9 |
+
* Model creator: [Sao10K](https://huggingface.co/Sao10K/)
|
10 |
+
* Original model: [Fimbulvetr-11B-v2](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)
|
11 |
+
|
12 |
+
<b>Important: </b> Inferencing for newer formats (IQ3_S, IQ4_NL, etc) was tested on llama.cpp. These newer quants may not work with other methods yet (as of 2/25/24).
|
13 |
|
14 |
|
15 |
All credits to Sao10K for the original model. This is just a quick test of the new quantization types such as IQ_3S in an attempt to further reduce VRAM requirements.
|
16 |
|
17 |
Quantized from fp16 with love. Importance matrix file [Fimbulvetr-11B-v2-imatrix.dat](https://huggingface.co/InferenceIllusionist/Fimbulvetr-11B-v2-iMat-GGUF/blob/main/Fimbulvetr-11B-v2-imatrix.dat) was calculated using Q8_0.
|
18 |
|
|
|
19 |
|
20 |
+
<i>Looking for Q3/Q4/Q5 quants? See the link in the original model card below.</i>
|
21 |
|
22 |
---
|
23 |
|
|
|
28 |
|
29 |
**https://huggingface.co/Sao10K/Fimbulvetr-11B-v2-GGUF <------ GGUF**
|
30 |
|
31 |
+
# Fimbulvetr-v2 - A Solar-Based Model
|
32 |
|
33 |
Prompt Formats - Alpaca or Vicuna. Either one works fine.
|
34 |
Recommended SillyTavern Presets - Universal Light
|