Commit History

Update chart
a57f878

5w4n commited on

Add new sample button
f25131d

5w4n commited on

Add cohere multilingual v3
bfa0a38

5w4n commited on

Add cohere multilingual tokenizer
fc19172

5w4n commited on

Add simbolo tokenizers
52f4fdb

5w4n commited on

Add simbolo tokenizer
75515fa

5w4n commited on

Update emoji
5dcfc72

5w4n commited on

Update about the project
45abc63

5w4n commited on

Add error handling for no tokenizer selected
9acdffa

5w4n commited on

Add token distribution chart and token count variability
6841f1c

5w4n commited on

Format the output
0158b9f

5w4n commited on

Fix bug
3e67b95

5w4n commited on

Make the code to use precalculated tokens from the dataset
34eb18c

5w4n commited on

Move the token results from sidebar
116574f

5w4n commited on

Update dataset
b1e9d2d

5w4n commited on

Add requirements
397a76e

5w4n commited on

Init
daa27ce

5w4n commited on

Add dataset
238ae95

5w4n commited on

initial commit
79a579c
verified

5w4n commited on