Can't load the model into llama.cpp
#4
by
ikcikoR
- opened
I'm getting the following error:
llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 642, got 514
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model '../models/ggml-c4ai-command-r-plus-104b-iq2_xxs.gguf'
{"tid":"140149869789376","timestamp":1712538037,"level":"ERR","function":"load_model","line":681,"msg":"unable to load model","model":"../models/ggml-c4ai-command-r-plus-104b-iq2_xxs.gguf"}```
When trying to run the model with this command `./server -m ../models/ggml-c4ai-command-r-plus-104b-iq2_xxs.gguf -ngl 20 --host
0.0.0.0 -c 4096`
Other AI models work properly but the 3 different GGUF sizes I tried downloading from this repo don't. I'm using latest git commit of llama.cpp.
Any idea what is causing it or if it can be fixed?
Nvm apparently the support isn't merged in yet.
ikcikoR
changed discussion status to
closed