Commit
•
c4958c0
1
Parent(s):
06c5e43
Fix typo (#2)
Browse files- Fix typo (bd46c0e2f8b8e640d0b6d9b38cb6d945a70b7dbe)
Co-authored-by: Sakasegawa <[email protected]>
README.md
CHANGED
|
@@ -82,7 +82,7 @@ for gen_text in tokenizer.batch_decode(gen_tokens, skip_special_tokens=True):
|
|
| 82 |
The model was trained on [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz), [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch), and [Japanese OSCAR](https://huggingface.co/datasets/oscar).
|
| 83 |
|
| 84 |
# Tokenization
|
| 85 |
-
The model uses a [special sub-word tokenizer](https://github.com/tanreinama/Japanese-BPEEncoder_V2). Please refer the original repository or [GPT-
|
| 86 |
|
| 87 |
# Licenese
|
| 88 |
[The MIT license](https://opensource.org/licenses/MIT)
|
|
|
|
| 82 |
The model was trained on [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz), [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch), and [Japanese OSCAR](https://huggingface.co/datasets/oscar).
|
| 83 |
|
| 84 |
# Tokenization
|
| 85 |
+
The model uses a [special sub-word tokenizer](https://github.com/tanreinama/Japanese-BPEEncoder_V2). Please refer the original repository or [GPT-NeoX-Japanese](https://huggingface.co/docs/transformers/model_doc/gpt_neox_japanese) in detail.
|
| 86 |
|
| 87 |
# Licenese
|
| 88 |
[The MIT license](https://opensource.org/licenses/MIT)
|