Hugging Face's logo

kromquant
/
L3.1-Tivir-10B-GGUFs

Transformers
GGUF
Inference Endpoints
conversational
Model card Files Community
L3.1-Tivir-10B-GGUFs / README.md
kromeurus's picture
kromeurus
Update README.md
4ac7ecb verified
preview code
|
raw
history blame
593 Bytes
metadata
base_model: kromeurus/L3.1-Tivir-v0.1-10B
library_name: transformers
quantized_by: kromeurus

This repo only contains the Q8, Q6, Q5, & Q4 GGUF files of Tivir v0.1

For the details of this model, please refer to the orginal model card here.

Additionally, there is another Q_8 quant in this repo named 'fQ_8'. It's not a new type of quant, rather a Q_8 quant that has the full float32 tokenizer. It's slightly larger then a normal Q_8 and meant to be an alternative quant. If it's giving you shitty outputs, use the normal Q_8.