Changes for fast tokenizer

#7

this seems ready no?

jonatanklosko changed pull request status to open

@ArthurZ yeah, I forgot to undraft this one!

ArthurZ changed pull request status to merged

Sign up or log in to comment