トークナイザーに全角記号・数字のNKFC正規化を追加

by misdelivery - opened Sep 2

←

Sep 2

•

トークナイザーに全角記号・数字のNKFC正規化を追加。

misdelivery changed pull request title from Upload tokenizer.json to トークナイザーに全角記号・数字のNKFC正規化を追加 Sep 2

kanhatakeyama changed pull request status to merged Sep 2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment