File size: 457 Bytes
f39d848
 
a1cc5bb
 
f39d848
 
a1cc5bb
f39d848
a1cc5bb
f39d848
a1cc5bb
f39d848
a1cc5bb
1
2
3
4
5
6
7
8
9
10
11
12
13
---
library_name: transformers
language:
- ms
---

# Malaysian Mistral 474M on MLM task using 512 context length

Replicating https://github.com/McGill-NLP/llm2vec using https://huggingface.co/mesolitica/malaysian-mistral-474M-4096, done by https://github.com/aisyahrzk https://twitter.com/aisyahhhrzk

Source code at https://github.com/mesolitica/malaya/tree/master/session/llm2vec

WandB, https://wandb.ai/aisyahrazak/mistral-474M-mlm?nw=nwuseraisyahrazak