not a string
#51
by
zhouxiaoou
- opened
llama_tokenizer3 = LlamaTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct", use_auth_token=token)
File "/Users/chaozhou/anaconda3/envs/my_llama_test_env/lib/python3.10/site-packages/sentencepiece/init.py", line 310, in LoadFromFile
return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
TypeError: not a string
amzing，use AutoTokenizer.from_pretrained(model_path, use_auth_token=token) don not have the error!!