unsloth

/

QwQ-32B-GGUF

like 47

Follow

Unsloth AI 3.59k

I tried deploying QwQ-32B-Q4_K_M with Ollama, but it keeps generating an endless loop of self-asked and self-answered responses. Has anyone encountered the same issue?

Here is the configuration I used to import the model:

FROM ./QwQ-32B-Q4_K_M.gguf
PARAMETER temperature 0.6
PARAMETER min_p 0.00
PARAMETER repeat_penalty 1.0
PARAMETER top_k 40
PARAMETER top_p 0.95
PARAMETER num_predict 32768
STOP "<|im_start|>"
STOP "<|im_end|>"