dariolopez/Llama-2-databricks-dolly-oasst1-es-axolotl-GGUF

Llama 2 (7B) fine-tuned on a own Spanish instructions dataset.

On this repo you can find 4-bit and 5-bit quantized versions of the Llama 2 (7B) Spanish fine-tuned.

How to use

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && git pull && make clean && make
git clone https://huggingface.co/dariolopez/Llama-2-databricks-dolly-oasst1-es-axolotl-GGUF
./main -m ./llama-2-databricks-dolly-oasst1-es-axolotl.gguf.q4_k_m.bin -n 2048 --color --temp 0 -ngl 35 -p "<s>[INST] Describe 5 lugares para visitar en España: [/INST]"

Based on

https://mlabonne.github.io/blog/posts/Quantize_Llama_2_models_using_ggml.html