KeyError: '<|endoftext|>' when using the tokenizer
#3
by
YellowDoge
- opened
Failed to initialize the tokenizer. Is it related to the version?
I also counter the same problem, have you solved it ?
Same problem here. Hopefully anyone can fix it
Same problem +1
更新下transformers
按照依赖安装,截图中的问题已经解决,更新一下模型实现的文件
pip install --upgrade transformers
solved my problem
pip install transformers==4.40.0
zRzRzRzRzRzRzR
changed discussion status to
closed
It seems this solution cannot work after installing 4.40.0. Use
pip install transformers --upgrade
can help.