Yuta Hayashibe
commited on
Commit
·
5efdce7
1
Parent(s):
3dd9d27
Update README.md
Browse files
README.md
CHANGED
|
@@ -22,7 +22,7 @@ Training codes are [available on GitHub](https://github.com/megagonlabs/t5-japan
|
|
| 22 |
- Japanese in [mC4/3.0.1](https://github.com/allenai/allennlp/discussions/5056)
|
| 23 |
- [Japanese in wiki40b/1.3.0](https://www.tensorflow.org/datasets/catalog/wiki40b#wiki40bja)
|
| 24 |
|
| 25 |
-
###
|
| 26 |
|
| 27 |
SentencePiece trained on Japanese Wikipedia
|
| 28 |
|
|
@@ -34,6 +34,8 @@ SentencePiece trained on Japanese Wikipedia
|
|
| 34 |
- T5 model: [models/t5.1.1.base.gin](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/t5/models/gin/models/t5.1.1.base.gin)
|
| 35 |
- Training steps: 1,000,000
|
| 36 |
|
|
|
|
|
|
|
| 37 |
## Related models
|
| 38 |
|
| 39 |
- [日本語T5事前学習済みモデル (sonoisa/t5-base-japanese)](https://huggingface.co/sonoisa/t5-base-japanese)
|
|
|
|
| 22 |
- Japanese in [mC4/3.0.1](https://github.com/allenai/allennlp/discussions/5056)
|
| 23 |
- [Japanese in wiki40b/1.3.0](https://www.tensorflow.org/datasets/catalog/wiki40b#wiki40bja)
|
| 24 |
|
| 25 |
+
### Tokenizer
|
| 26 |
|
| 27 |
SentencePiece trained on Japanese Wikipedia
|
| 28 |
|
|
|
|
| 34 |
- T5 model: [models/t5.1.1.base.gin](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/t5/models/gin/models/t5.1.1.base.gin)
|
| 35 |
- Training steps: 1,000,000
|
| 36 |
|
| 37 |
+
It took about 126 hours with TPU v3-8
|
| 38 |
+
|
| 39 |
## Related models
|
| 40 |
|
| 41 |
- [日本語T5事前学習済みモデル (sonoisa/t5-base-japanese)](https://huggingface.co/sonoisa/t5-base-japanese)
|