Feedback / ChatML format issues.
This ain't the test I promised because I hit this recurrent issue with your model:
Sometimes, it tries to add "<|user|>" and "<|system|>" fields to its response (and yes, it's adding stuff that would fit user or system info after that). It's relatively rare, but it sounds like something went wrong with the formatting at some point during the merge or the tuning of the parent models. If it was "<|im_start|>user" or "<|im_start|>system" I would at least get it, but here, I'm not sure. That would suggest that it's not trained on ChatML, but on whatever this is instead.
Any idea?
I've not seen that, but once in a blue moon I've seen an output seemingly stall before ending. Maybe my front-end is hiding it from me. Not all the merge components were trained for ChatML, which might explain why the problem is happening only occasionally. It's possible fine-tuning could address this.
Will test out Alpaca format later.
I've not seen that, but once in a blue moon I've seen an output seemingly stall before ending. Maybe my front-end is hiding it from me. Not all the merge components were trained for ChatML, which might explain why the problem is happening only occasionally. It's possible fine-tuning could address this.
Will test out Alpaca format later.
Seems to be Zephyr's prompt "<|user|>\n ... <|system|>" and i found this relating to cut off messages. "Truncating Response"
obligatory
To be fair, most of the time it works, it's relatively easy to filter out, and the model is still quite good (in my super official first test). Also it's the first mistral model for which I don't have to push the Rep.Pen. a ton to get decently written and non repetitive outputs.
My personal take on this model is that it's a mildly unstable merge, but the tradeoff is improved creativity.