musiclang-4k / tokenizer_config.json
floriangardin's picture
Create tokenizer_config.json
968db24
raw
history blame
154 Bytes
{
"model_max_len": 4096,
"pad_token": "WILL_END",
"pad_token_id": 65,
"eos_token": "WILL_END",
"eos_token_id": 65,
"add_prefix_space": false
}