model seems to output only gibberish

#1
by JoggyMuffin - opened

I've tried the recommended context settings in the original page and a bunch of different sampler settings in sillytavern but this model seems dead set on only outputting complete nonsense and broken english only loosely related to the input.

I'm using i1-Q4_K_S on sillytavern with the latest version of ooba main

Update: I'm not sure if it's something with the model settings that got imported or if ooba was being weird but the model imported with a rope_freq_base of something like 5 million. I reset it to the 10k recommended on the original model page and it's working much more coherently now.

I downloaded the Q4_K_S. It has rope settings of 1/10000, and works out of the box that way with koboldcpp, so it seems this is not an issue with the model itself.

mradermacher changed discussion status to closed

Sign up or log in to comment