mesolitica
/

malaysian-debertav2-base

Inference Endpoints

Model card Files Files and versions Community

Edit model card

Pretrain BASE 512 masking context length DebertaV2 on Malaysian text

Special thanks to https://github.com/aisyahrzk for pretraining DebertaV2 Base.

WanDB at https://wandb.ai/aisyahrazak/deberta-base?nw=nwuseraisyahrazak

Downloads last month: 15

Safetensors

Model size

114M params

Tensor type

BF16

·

Inference API

Unable to determine this model’s pipeline type. Check the docs .

Collection including mesolitica/malaysian-debertav2-base

Malaysian MaskLM

Trained on 17B tokens, 81GB of cleaned texts, able to understand standard Malay, local Malay, local Mandarin, Manglish, and local Tamil. • 7 items • Updated Sep 22