YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Should be working properly with KoboldCPP and other stuff. I changed EOS token to <|eot_id|> and this prevents endless generation.

I also converted original weights back to fp32 before quantization.

Edit: Highly recommending you to download more recent quants. 17/05/2024

GGUF

Model size

8.03B params

Architecture

llama

6-bit

8-bit

Inference API

Unable to determine this model's library. Check the docs .