|
--- |
|
language: da |
|
widget: |
|
- text: En trend, der kan blive ligeså hot som<mask>. |
|
tags: |
|
- roberta |
|
- danish |
|
- masked-lm |
|
- pytorch |
|
license: cc-by-4.0 |
|
--- |
|
|
|
# DanskBERT |
|
|
|
This is DanskBERT, a Danish language model. Note that you should not prepend the mask with a space when using it directly! |
|
|
|
The model is the best performing base-size model on the [ScandEval benchmark for Danish](https://scandeval.github.io/nlu-benchmark/). |
|
|
|
DanskBERT was trained on the Danish Gigaword Corpus (Strømberg-Derczynski et al., 2021). |
|
|
|
DanskBERT was trained using fairseq using the RoBERTa-base configuration. The model was trained with a batch size of 2k, and was trained to convergence for 500k steps using 16 V100 cards for approximately two weeks. |
|
|
|
If you find this model useful, please cite |
|
|
|
``` |
|
@inproceedings{snaebjarnarson-etal-2023-transfer, |
|
title = "{T}ransfer to a Low-Resource Language via Close Relatives: The Case Study on Faroese", |
|
author = "Snæbjarnarson, Vésteinn and |
|
Simonsen, Annika and |
|
Glavaš, Goran and |
|
Vulić, Ivan", |
|
booktitle = "Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)", |
|
month = "may 22--24", |
|
year = "2023", |
|
address = "Tórshavn, Faroe Islands", |
|
publisher = {Link{\"o}ping University Electronic Press, Sweden}, |
|
} |
|
``` |
