Tokenizer difference

#1
by dranger003 - opened

Thanks for this great model! It is also great to see support in llama.cpp for it.

At this point however there seems to be an issue with the tokenizer - more details here:
https://github.com/ggerganov/llama.cpp/issues/5496

Edit: I just realised Cebtenzzre is part of Nomic, so I guess you already know about this then :)

great, I just experience this issue

Sign up or log in to comment