Hugging Face's logo

Gameselo
/
STS-multilingual-mpnet-base-v2

Sentence Similarity
sentence-transformers
Safetensors
xlm-roberta
mteb
feature-extraction
dataset_size:100K<n<1M
loss:AnglELoss
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
1
Gameselo commited on
Commit
1ffb8e9
•
1 Parent(s): dc9447b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +33 -30
README.md CHANGED
@@ -995,37 +995,40 @@ model-index:
995
  name: Spearman Max
996
  ---
997
 
998
- /!\ This model achieves SOTA results in the MTEB STS multilingual Leaderboard (in "other"). Here is the comparison
999
 
1000
- State-of-the-art results (Multi) STSb-XLM-RoBERTa-base Paraphrase Multilingual MPNet base v2
1001
- Average 73.17 71.68 **73.89**
1002
- STS17 (ar-ar) **81.87** 80.43 81.24
1003
- STS17 (en-ar) **81.22** 76.3 77.03
1004
- STS17 (en-de) 87.3 91.06 **91.09**
1005
- STS17 (en-tr) 77.18 **80.74** 79.87
1006
- STS17 (es-en) **88.24** 83.09 85.53
1007
- STS17 (es-es) **88.25** 84.16 87.27
1008
- STS17 (fr-en) 88.06 **91.33** 90.68
1009
- STS17 (it-en) 89.68 **92.87** 92.47
1010
- STS17 (ko-ko) 83.69 **97.67** 97.66
1011
- STS17 (nl-en) 88.25 **92.13** 91.15
1012
- STS22 (ar) 58.67 58.67 **62.66**
1013
- STS22 (de) **60.12** 52.17 57.74
1014
- STS22 (de-en) **60.92** 58.5 57.5
1015
- STS22 (de-fr) **67.79** 51.28 57.99
1016
- STS22 (de-pl) **58.69** 44.56 44.22
1017
- STS22 (es) **68.57** 63.68 66.21
1018
- STS22 (es-en) **78.8** 70.65 75.18
1019
- STS22 (es-it) **75.04** 60.88 66.25
1020
- STS22 (fr) **83.75** 76.46 78.76
1021
- STS22 (fr-pl) 84.52 84.52 **84.52**
1022
- STS22 (it) **79.28** 66.73 68.47
1023
- STS22 (pl) 42.08 41.18 **43.36**
1024
- STS22 (pl-en) **77.5** 64.35 75.11
1025
- STS22 (ru) **61.71** 58.59 58.67
1026
- STS22 (tr) **68.72** 57.52 63.84
1027
- STS22 (zh-en) **71.88** 60.69 65.37
1028
- STSb 89.86 95.05 **95.15**
 
 
 
1029
 
1030
  # SentenceTransformer based on sentence-transformers/paraphrase-multilingual-mpnet-base-v2
1031
 
 
995
  name: Spearman Max
996
  ---
997
 
998
+ ## State-of-the-Art Results Comparison (MTEB STS Multilingual Leaderboard)
999
 
1000
+ | Dataset | State-of-the-art (Multi) | STSb-XLM-RoBERTa-base | Paraphrase Multilingual MPNet base v2 |
1001
+ |-------------------|--------------------------|-----------------------|--------------------------------------|
1002
+ | Average | 73.17 | 71.68 | **73.89** |
1003
+ | STS17 (ar-ar) | **81.87** | 80.43 | 81.24 |
1004
+ | STS17 (en-ar) | **81.22** | 76.3 | 77.03 |
1005
+ | STS17 (en-de) | 87.3 | 91.06 | **91.09** |
1006
+ | STS17 (en-tr) | 77.18 | **80.74** | 79.87 |
1007
+ | STS17 (es-en) | **88.24** | 83.09 | 85.53 |
1008
+ | STS17 (es-es) | **88.25** | 84.16 | 87.27 |
1009
+ | STS17 (fr-en) | 88.06 | **91.33** | 90.68 |
1010
+ | STS17 (it-en) | 89.68 | **92.87** | 92.47 |
1011
+ | STS17 (ko-ko) | 83.69 | **97.67** | 97.66 |
1012
+ | STS17 (nl-en) | 88.25 | **92.13** | 91.15 |
1013
+ | STS22 (ar) | 58.67 | 58.67 | **62.66** |
1014
+ | STS22 (de) | **60.12** | 52.17 | 57.74 |
1015
+ | STS22 (de-en) | **60.92** | 58.5 | 57.5 |
1016
+ | STS22 (de-fr) | **67.79** | 51.28 | 57.99 |
1017
+ | STS22 (de-pl) | **58.69** | 44.56 | 44.22 |
1018
+ | STS22 (es) | **68.57** | 63.68 | 66.21 |
1019
+ | STS22 (es-en) | **78.8** | 70.65 | 75.18 |
1020
+ | STS22 (es-it) | **75.04** | 60.88 | 66.25 |
1021
+ | STS22 (fr) | **83.75** | 76.46 | 78.76 |
1022
+ | STS22 (fr-pl) | 84.52 | 84.52 | **84.52** |
1023
+ | STS22 (it) | **79.28** | 66.73 | 68.47 |
1024
+ | STS22 (pl) | 42.08 | 41.18 | **43.36** |
1025
+ | STS22 (pl-en) | **77.5** | 64.35 | 75.11 |
1026
+ | STS22 (ru) | **61.71** | 58.59 | 58.67 |
1027
+ | STS22 (tr) | **68.72** | 57.52 | 63.84 |
1028
+ | STS22 (zh-en) | **71.88** | 60.69 | 65.37 |
1029
+ | STSb | 89.86 | 95.05 | **95.15** |
1030
+
1031
+ **Bold** indicates the best result in each row.
1032
 
1033
  # SentenceTransformer based on sentence-transformers/paraphrase-multilingual-mpnet-base-v2
1034