|
--- |
|
license: cc-by-4.0 |
|
language: |
|
- en |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
Just for fun, I tried to create an imatrix model for Kaiju-11B. (https://huggingface.co/Himitsui/Kaiju-11B) |
|
|
|
I thought it wouldn’t work, since I only have a laptop with an Nvidia 3060 with 6GB of memory, but strangely enough, I was able to create a couple of models thanks to one script. |
|
|
|
Here it is: https://huggingface.co/FantasiaFoundry/GGUF-Quantization-Script |
|
|
|
According to the recommendations, my laptop was not suitable. I don’t know how it all works, maybe these were just recommendations to make quantization happen quickly. (it took me about an hour and a half to create the imatrix.bat file, but the quantization was fast) |
|
If anyone is interested, download and see how they work, this was done purely for fun, no more no less. |
