42dot_LLM-PLM-1.3B / added_tokens.json
ykhwang's picture
Upload tokenizer
5cad9c2
raw
history blame
95 Bytes
{
"<|endoftext|>": 50256,
"<||bos||>": 50257,
"<||pad||>": 50258,
"<||unk||>": 50259
}