I would like to automatically reconcile the correct tokenizer for this model without having to implement custom logic in my code. Could you please add a tokenizer field to your config.json? Thanks!
tokenizer
· Sign up or log in to comment