Please also upload the tokenizers.model(sentencepiece tokenizer) file because I would like to extend the smollm2-360m model to a non-english language, but only sentencepiece tokenizer can be extended & not huggingface tokenizers(tokenizers.json)
· Sign up or log in to comment
