BlueNipples
commited on
Commit
•
4d1b2fa
1
Parent(s):
2f9f776
Update README.md
Browse files
README.md
CHANGED
@@ -7,10 +7,14 @@ tags:
|
|
7 |
---
|
8 |
![SnowLotus Logo](https://cdn-uploads.huggingface.co/production/uploads/64bb1109aaccfd28b023bcec/gTQtPK46laLIFg0RTAv73.png)
|
9 |
|
|
|
|
|
10 |
3x Importance Matrix GGUFs and 2x regular GGUFs for https://huggingface.co/BlueNipples/SnowLotus-v2-10.7B and https://huggingface.co/BlueNipples/DaringLotus-v2-10.7b.
|
11 |
|
12 |
I'm super happy with these merges, they turned out great. Basically Daring is the slightly more creative/prose oriented one, but also slightly less coherent. They both have excellent prose for their size that is largely not very gpt-ish and are able to often take story context, lore entries and character card info into account. You can probably use these as your mainstay - which especially helpful if you GPU struggles with 13b, and honestly I think these models are _probably_ equal to or better than any 13b anyway. I might be wrong, but I do think they are very good compared to anything I've personally run. See the individual model cards for merge recipe details.
|
13 |
|
14 |
Thanks to lucyknada for helping me get the imatrix quants done quicker!
|
15 |
|
16 |
-
|
|
|
|
|
|
7 |
---
|
8 |
![SnowLotus Logo](https://cdn-uploads.huggingface.co/production/uploads/64bb1109aaccfd28b023bcec/gTQtPK46laLIFg0RTAv73.png)
|
9 |
|
10 |
+
## Summary
|
11 |
+
|
12 |
3x Importance Matrix GGUFs and 2x regular GGUFs for https://huggingface.co/BlueNipples/SnowLotus-v2-10.7B and https://huggingface.co/BlueNipples/DaringLotus-v2-10.7b.
|
13 |
|
14 |
I'm super happy with these merges, they turned out great. Basically Daring is the slightly more creative/prose oriented one, but also slightly less coherent. They both have excellent prose for their size that is largely not very gpt-ish and are able to often take story context, lore entries and character card info into account. You can probably use these as your mainstay - which especially helpful if you GPU struggles with 13b, and honestly I think these models are _probably_ equal to or better than any 13b anyway. I might be wrong, but I do think they are very good compared to anything I've personally run. See the individual model cards for merge recipe details.
|
15 |
|
16 |
Thanks to lucyknada for helping me get the imatrix quants done quicker!
|
17 |
|
18 |
+
## Importance Matrix Note
|
19 |
+
|
20 |
+
Imatrix currently does not run with Koboldcpp although bound to tbe supported in the future as it is supported by Llamacpp (and I'm guessing therefor ooba). Those quants should provide a perplexity boost especially to the smaller quants. The dat files are also there so if you make a fp16 gguf from the main model cards you might be able to save yourself some time producing your own imatrix quants.
|