|
--- |
|
language: |
|
- sk |
|
license: cc |
|
tags: |
|
- sentence-transformers |
|
- feature-extraction |
|
- sentence-similarity |
|
- sts |
|
datasets: |
|
- nyu-mll/glue |
|
metrics: |
|
- spearmanr |
|
pipeline_tag: sentence-similarity |
|
widget: |
|
- source_sentence: Izrael uskutočnil letecké údery v blízkosti Damasku. |
|
sentences: |
|
- Izrael uskutočnil vzdušný útok na Sýriu. |
|
- Pes leží na gauči a má hlavu na bielom vankúši. |
|
--- |
|
|
|
|
|
# Sentence similarity model based on SlovakBERT |
|
|
|
This is a sentence similarity model based on [SlovakBERT](https://huggingface.co/gerulata/slovakbert). The model was fine-tuned using [STSbenchmark](https://ixa2.si.ehu.eus/stswiki/index.php/STSbenchmark) [Cer et al 2017] translated to Slovak using [M2M100](https://huggingface.co/facebook/m2m100_1.2B). The model can be used as an universal sentence encoder for Slovak sentences. |
|
|
|
## Results |
|
|
|
The model was evaluated in [our paper](https://arxiv.org/abs/2109.15254) [Pikuliak et al 2021, Section 4.3]. It achieves \\(0.791\\) Spearman correlation on STSbenchmark test set. |
|
|
|
|
|
## Usage |
|
|
|
Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed: |
|
|
|
``` |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can use the model like this: |
|
|
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
sentences = ["This is an example sentence", "Each sentence is converted"] |
|
|
|
model = SentenceTransformer('kinit/slovakbert-sts-stsb') |
|
embeddings = model.encode(sentences) |
|
print(embeddings) |
|
``` |
|
|
|
|
|
## Cite |
|
|
|
``` |
|
@inproceedings{pikuliak-etal-2022-slovakbert, |
|
title = "{S}lovak{BERT}: {S}lovak Masked Language Model", |
|
author = "Pikuliak, Mat{\'u}{\v{s}} and |
|
Grivalsk{\'y}, {\v{S}}tefan and |
|
Kon{\^o}pka, Martin and |
|
Bl{\v{s}}t{\'a}k, Miroslav and |
|
Tamajka, Martin and |
|
Bachrat{\'y}, Viktor and |
|
Simko, Marian and |
|
Bal{\'a}{\v{z}}ik, Pavol and |
|
Trnka, Michal and |
|
Uhl{\'a}rik, Filip", |
|
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2022", |
|
month = dec, |
|
year = "2022", |
|
address = "Abu Dhabi, United Arab Emirates", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://aclanthology.org/2022.findings-emnlp.530", |
|
pages = "7156--7168", |
|
abstract = "We introduce a new Slovak masked language model called \textit{SlovakBERT}. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.", |
|
} |
|
``` |
|
|
