Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
Jaward 
posted an update May 23
Post
992
mlx port of karpathy’s minbpe 🤕
Minimal (byte-level) Byte Pair Encoding tokenizer. Algorithmically follows along the GPT2 tokenizer.
Code: https://github.com/Jaykef/mlx-minbpe
In this post