This repo only contains the Q8, Q6, Q5, & Q4 GGUF files of Tivir v0.1

For the details of this model, please refer to the orginal model card here.

Additionally, there is another Q_8 quant in this repo named 'fQ_8'. It's not a new type of quant, rather a Q_8 quant that has the full float32 tokenizer. It's slightly larger then a normal Q_8 and meant to be an alternative quant. If it's giving you shitty outputs, use the normal Q_8.

Downloads last month: 237

GGUF

Model size

10.6B params

Architecture

llama

3-bit

4-bit

5-bit

6-bit

8-bit

32-bit

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model’s pipeline type.

Model tree for kromquant/L3.1-Tivir-10B-GGUFs

Base model

kromeurus/L3.1-Tivir-v0.1-10B

Quantized

(3)

this model