pokemon / tokenizer

Commit History

Rename tokenizer/tokenizer_vocab.json to tokenizer/vocab.json
bdeabc2
verified

vuongminhkhoi4 commited on

Rename tokenizer/tokenizer_tokenizer_config.json to tokenizer/tokenizer_config.json
abc1085
verified

vuongminhkhoi4 commited on

Rename tokenizer/tokenizer_special_tokens_map.json to tokenizer/special_tokens_map.json
d64824e
verified

vuongminhkhoi4 commited on

Rename tokenizer/tokenizer_merges.txt to tokenizer/merges.txt
f8cc00b
verified

vuongminhkhoi4 commited on