meelu
/

DA-BPE-LLAMA3.2-TOKEN

BPE Tokenization

Model card Files Files and versions Community

DA-BPE-LLAMA3.2-TOKEN / special_tokens_map.json

MikkelWK's picture

Rename DA BPE Llama 3.2 Special Tokens Map.json to special_tokens_map.json

5c50ea3 verified 23 days ago

141 Bytes

	{
	"pad_token": {
	"content": "[PAD]",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false
	}
	}