Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
pietrolesci
/
raw_tokenisers
like
0
Model card
Files
Files and versions
Community
72714a5
raw_tokenisers
File size: 66 Bytes
72714a5
1
2
3
4
max_vocab_size:
320000
num_docs:
1000000
eos_token:
<|endoftext|>
