File size: 671 Bytes
4f948a1 c2b337c 3efa064 0d157f6 4f948a1 c2b337c 77aa4f2 c2b337c 77aa4f2 c2b337c 6de87a0 c2b337c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
---
license: other
datasets:
- facebook/belebele
---
Pretrained toy models. Made with Andrej Karpathy's NanoGPT.
# nano_35m
* Trained late 2023 on part of Tagalog portion of Belebele.
* batch_size = 64
* block_size = 256
* n_layer = 8
* n_head = 8
* n_embd = 768
* Everything else is left as is.
# nano_76m
* Trained January 2024 on part of Tagalog portion of Belebele.
* batch_size = 64
* block_size = 256
* n_layer = 11
* n_head = 16
* n_embd = 768
* Everything else is left as is.
# nano-ito_35m
* Trained March 2024 on part of PALITO Tagalog dataset.
* batch_size = 64
* block_size = 256
* n_layer = 11
* n_head = 16
* n_embd = 512
* Everything else is left as is. |