meta-llama

/

Meta-Llama-3-8B

like 5.82k

Follow

Meta Llama 7,787

When I wanted to quantify the fine-tuned model(llama3 8b), I ran into this problem。
command:
sudo ./llama-quantize /root/hg_to_gguf.gguf/Llama_Lora_Merge-8.0B-F16.gguf /root/quan
tize_model q4_0
problem:
llama_model_quantize: failed to quantize: basic_ios::clear: iostream error
main: failed to quantize model from '/root/hg_to_gguf.gguf/Llama_Lora_Merge-8.0B-F16.gguf'