diff --git "a/README.md" "b/README.md" new file mode 100644--- /dev/null +++ "b/README.md" @@ -0,0 +1,9028 @@ +--- +library_name: model2vec +license: mit +model-index: +- name: potion-base-4M + results: + - dataset: + config: en-ext + name: MTEB AmazonCounterfactualClassification (en-ext) + revision: e8379541af4e31359cca9fbcf4b00f2671dba205 + split: test + type: mteb/amazon_counterfactual + metrics: + - type: accuracy + value: 67.5712143928036 + - type: ap + value: 17.612722401291446 + - type: ap_weighted + value: 17.612722401291446 + - type: f1 + value: 55.192880430611815 + - type: f1_weighted + value: 73.78539599566238 + - type: main_score + value: 67.5712143928036 + task: + type: Classification + - dataset: + config: en + name: MTEB AmazonCounterfactualClassification (en) + revision: e8379541af4e31359cca9fbcf4b00f2671dba205 + split: test + type: mteb/amazon_counterfactual + metrics: + - type: accuracy + value: 68.38805970149254 + - type: ap + value: 29.77595242354233 + - type: ap_weighted + value: 29.77595242354233 + - type: f1 + value: 61.660550379147104 + - type: f1_weighted + value: 71.3687043183968 + - type: main_score + value: 68.38805970149254 + task: + type: Classification + - dataset: + config: default + name: MTEB AmazonPolarityClassification (default) + revision: e2d317d38cd51312af73b3d32a06d1a08b442046 + split: test + type: mteb/amazon_polarity + metrics: + - type: accuracy + value: 71.69602499999999 + - type: ap + value: 66.0912235632392 + - type: ap_weighted + value: 66.0912235632392 + - type: f1 + value: 71.42572451950106 + - type: f1_weighted + value: 71.42572451950106 + - type: main_score + value: 71.69602499999999 + task: + type: Classification + - dataset: + config: en + name: MTEB AmazonReviewsClassification (en) + revision: 1399c76144fd37290681b995c656ef9b2e06e26d + split: test + type: mteb/amazon_reviews_multi + metrics: + - type: accuracy + value: 34.321999999999996 + - type: f1 + value: 33.685910374348474 + - type: f1_weighted + value: 33.68591037434848 + - type: main_score + value: 34.321999999999996 + task: + type: Classification + - dataset: + config: default + name: MTEB ArguAna (default) + revision: c22ab2a51041ffd869aaddef7af8d8215647e41a + split: test + type: mteb/arguana + metrics: + - type: main_score + value: 39.449 + - type: map_at_1 + value: 19.203 + - type: map_at_10 + value: 32.068000000000005 + - type: map_at_100 + value: 33.315 + - type: map_at_1000 + value: 33.352 + - type: map_at_20 + value: 32.891 + - type: map_at_3 + value: 27.916 + - type: map_at_5 + value: 30.238 + - type: mrr_at_1 + value: 19.701280227596015 + - type: mrr_at_10 + value: 32.25501817607077 + - type: mrr_at_100 + value: 33.50123053139716 + - type: mrr_at_1000 + value: 33.53792294287078 + - type: mrr_at_20 + value: 33.08067358539894 + - type: mrr_at_3 + value: 28.0464675201517 + - type: mrr_at_5 + value: 30.375770507349387 + - type: nauc_map_at_1000_diff1 + value: 12.945790446862631 + - type: nauc_map_at_1000_max + value: 2.7562298858970746 + - type: nauc_map_at_1000_std + value: 6.308808342750878 + - type: nauc_map_at_100_diff1 + value: 12.950463273991264 + - type: nauc_map_at_100_max + value: 2.791124689377845 + - type: nauc_map_at_100_std + value: 6.355349751721058 + - type: nauc_map_at_10_diff1 + value: 12.520099598532141 + - type: nauc_map_at_10_max + value: 2.4000545594181673 + - type: nauc_map_at_10_std + value: 6.104620938201642 + - type: nauc_map_at_1_diff1 + value: 15.41697878849924 + - type: nauc_map_at_1_max + value: 0.06798318069793197 + - type: nauc_map_at_1_std + value: 2.287679644950582 + - type: nauc_map_at_20_diff1 + value: 12.955778688664418 + - type: nauc_map_at_20_max + value: 2.868790975264444 + - type: nauc_map_at_20_std + value: 6.391194651176906 + - type: nauc_map_at_3_diff1 + value: 13.201122451167164 + - type: nauc_map_at_3_max + value: 1.88922966188281 + - type: nauc_map_at_3_std + value: 4.49834374299309 + - type: nauc_map_at_5_diff1 + value: 12.823675090212786 + - type: nauc_map_at_5_max + value: 1.7834104088257399 + - type: nauc_map_at_5_std + value: 5.065324239535852 + - type: nauc_mrr_at_1000_diff1 + value: 11.326932411816648 + - type: nauc_mrr_at_1000_max + value: 2.1073908479584578 + - type: nauc_mrr_at_1000_std + value: 6.263379081100884 + - type: nauc_mrr_at_100_diff1 + value: 11.333524099616422 + - type: nauc_mrr_at_100_max + value: 2.142759269416129 + - type: nauc_mrr_at_100_std + value: 6.309554100448871 + - type: nauc_mrr_at_10_diff1 + value: 10.948665635179944 + - type: nauc_mrr_at_10_max + value: 1.7458485198283498 + - type: nauc_mrr_at_10_std + value: 6.054483809416018 + - type: nauc_mrr_at_1_diff1 + value: 13.365599306041101 + - type: nauc_mrr_at_1_max + value: 0.10132975428191564 + - type: nauc_mrr_at_1_std + value: 2.3692857303611032 + - type: nauc_mrr_at_20_diff1 + value: 11.362849419180831 + - type: nauc_mrr_at_20_max + value: 2.2368929793175263 + - type: nauc_mrr_at_20_std + value: 6.342198336503448 + - type: nauc_mrr_at_3_diff1 + value: 11.501128611140333 + - type: nauc_mrr_at_3_max + value: 1.0918091675463633 + - type: nauc_mrr_at_3_std + value: 4.31343316721122 + - type: nauc_mrr_at_5_diff1 + value: 11.066202871817653 + - type: nauc_mrr_at_5_max + value: 0.9506044147251881 + - type: nauc_mrr_at_5_std + value: 5.079518879246342 + - type: nauc_ndcg_at_1000_diff1 + value: 12.927162083028898 + - type: nauc_ndcg_at_1000_max + value: 4.360655362716004 + - type: nauc_ndcg_at_1000_std + value: 8.410996652509247 + - type: nauc_ndcg_at_100_diff1 + value: 13.058208468182912 + - type: nauc_ndcg_at_100_max + value: 5.462308667986846 + - type: nauc_ndcg_at_100_std + value: 9.825403964300818 + - type: nauc_ndcg_at_10_diff1 + value: 11.612557086246108 + - type: nauc_ndcg_at_10_max + value: 4.101264941720262 + - type: nauc_ndcg_at_10_std + value: 8.829257094057274 + - type: nauc_ndcg_at_1_diff1 + value: 15.41697878849924 + - type: nauc_ndcg_at_1_max + value: 0.06798318069793197 + - type: nauc_ndcg_at_1_std + value: 2.287679644950582 + - type: nauc_ndcg_at_20_diff1 + value: 13.311704630454063 + - type: nauc_ndcg_at_20_max + value: 6.006804756561238 + - type: nauc_ndcg_at_20_std + value: 10.007008078971948 + - type: nauc_ndcg_at_3_diff1 + value: 12.989623337852278 + - type: nauc_ndcg_at_3_max + value: 2.708080727538749 + - type: nauc_ndcg_at_3_std + value: 5.190385461099005 + - type: nauc_ndcg_at_5_diff1 + value: 12.324781485912697 + - type: nauc_ndcg_at_5_max + value: 2.5703939367879904 + - type: nauc_ndcg_at_5_std + value: 6.312549457167382 + - type: nauc_precision_at_1000_diff1 + value: 15.719224058263089 + - type: nauc_precision_at_1000_max + value: 38.39806367915726 + - type: nauc_precision_at_1000_std + value: 59.476183642948556 + - type: nauc_precision_at_100_diff1 + value: 16.09148045706227 + - type: nauc_precision_at_100_max + value: 35.51651818510163 + - type: nauc_precision_at_100_std + value: 48.94915670796286 + - type: nauc_precision_at_10_diff1 + value: 8.649466288211341 + - type: nauc_precision_at_10_max + value: 9.98754064777929 + - type: nauc_precision_at_10_std + value: 18.337504631774866 + - type: nauc_precision_at_1_diff1 + value: 15.41697878849924 + - type: nauc_precision_at_1_max + value: 0.06798318069793197 + - type: nauc_precision_at_1_std + value: 2.287679644950582 + - type: nauc_precision_at_20_diff1 + value: 16.643675133736345 + - type: nauc_precision_at_20_max + value: 21.841551146673815 + - type: nauc_precision_at_20_std + value: 27.39523828734291 + - type: nauc_precision_at_3_diff1 + value: 12.562160662569264 + - type: nauc_precision_at_3_max + value: 4.884608119320729 + - type: nauc_precision_at_3_std + value: 6.97802947416861 + - type: nauc_precision_at_5_diff1 + value: 11.050717696142218 + - type: nauc_precision_at_5_max + value: 4.718154851488971 + - type: nauc_precision_at_5_std + value: 9.813084435699531 + - type: nauc_recall_at_1000_diff1 + value: 15.719224058264036 + - type: nauc_recall_at_1000_max + value: 38.39806367915738 + - type: nauc_recall_at_1000_std + value: 59.47618364294785 + - type: nauc_recall_at_100_diff1 + value: 16.091480457062136 + - type: nauc_recall_at_100_max + value: 35.51651818510153 + - type: nauc_recall_at_100_std + value: 48.949156707963105 + - type: nauc_recall_at_10_diff1 + value: 8.649466288211405 + - type: nauc_recall_at_10_max + value: 9.987540647779353 + - type: nauc_recall_at_10_std + value: 18.337504631774873 + - type: nauc_recall_at_1_diff1 + value: 15.41697878849924 + - type: nauc_recall_at_1_max + value: 0.06798318069793197 + - type: nauc_recall_at_1_std + value: 2.287679644950582 + - type: nauc_recall_at_20_diff1 + value: 16.643675133736263 + - type: nauc_recall_at_20_max + value: 21.841551146673865 + - type: nauc_recall_at_20_std + value: 27.39523828734288 + - type: nauc_recall_at_3_diff1 + value: 12.56216066256926 + - type: nauc_recall_at_3_max + value: 4.884608119320747 + - type: nauc_recall_at_3_std + value: 6.978029474168629 + - type: nauc_recall_at_5_diff1 + value: 11.05071769614224 + - type: nauc_recall_at_5_max + value: 4.718154851488988 + - type: nauc_recall_at_5_std + value: 9.81308443569954 + - type: ndcg_at_1 + value: 19.203 + - type: ndcg_at_10 + value: 39.449 + - type: ndcg_at_100 + value: 45.461 + - type: ndcg_at_1000 + value: 46.438 + - type: ndcg_at_20 + value: 42.473 + - type: ndcg_at_3 + value: 30.824 + - type: ndcg_at_5 + value: 35.010999999999996 + - type: precision_at_1 + value: 19.203 + - type: precision_at_10 + value: 6.315999999999999 + - type: precision_at_100 + value: 0.911 + - type: precision_at_1000 + value: 0.099 + - type: precision_at_20 + value: 3.759 + - type: precision_at_3 + value: 13.086999999999998 + - type: precision_at_5 + value: 9.886000000000001 + - type: recall_at_1 + value: 19.203 + - type: recall_at_10 + value: 63.158 + - type: recall_at_100 + value: 91.11 + - type: recall_at_1000 + value: 98.791 + - type: recall_at_20 + value: 75.178 + - type: recall_at_3 + value: 39.26 + - type: recall_at_5 + value: 49.431000000000004 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ArxivClusteringP2P (default) + revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d + split: test + type: mteb/arxiv-clustering-p2p + metrics: + - type: main_score + value: 34.19332453253303 + - type: v_measure + value: 34.19332453253303 + - type: v_measure_std + value: 14.189152053888938 + task: + type: Clustering + - dataset: + config: default + name: MTEB ArxivClusteringS2S (default) + revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 + split: test + type: mteb/arxiv-clustering-s2s + metrics: + - type: main_score + value: 24.104443846867934 + - type: v_measure + value: 24.104443846867934 + - type: v_measure_std + value: 15.164667270049211 + task: + type: Clustering + - dataset: + config: default + name: MTEB AskUbuntuDupQuestions (default) + revision: 2000358ca161889fa9c082cb41daa8dcfb161a54 + split: test + type: mteb/askubuntudupquestions-reranking + metrics: + - type: main_score + value: 53.803562861722696 + - type: map + value: 53.803562861722696 + - type: mrr + value: 67.899683419074 + - type: nAUC_map_diff1 + value: 5.929761636502396 + - type: nAUC_map_max + value: 16.36818121973646 + - type: nAUC_map_std + value: 3.3645669952351964 + - type: nAUC_mrr_diff1 + value: 10.961094805507717 + - type: nAUC_mrr_max + value: 21.221447117685774 + - type: nAUC_mrr_std + value: 5.470983299654257 + task: + type: Reranking + - dataset: + config: default + name: MTEB BIOSSES (default) + revision: d3fb88f8f02e40887cd149695127462bbcf29b4a + split: test + type: mteb/biosses-sts + metrics: + - type: cosine_pearson + value: 75.18540989714172 + - type: cosine_spearman + value: 71.87392685280749 + - type: euclidean_pearson + value: 73.96944357814664 + - type: euclidean_spearman + value: 71.87392685280749 + - type: main_score + value: 71.87392685280749 + - type: manhattan_pearson + value: 74.08100378291418 + - type: manhattan_spearman + value: 72.19080154289176 + - type: pearson + value: 75.18540989714172 + - type: spearman + value: 71.87392685280749 + task: + type: STS + - dataset: + config: default + name: MTEB Banking77Classification (default) + revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 + split: test + type: mteb/banking77 + metrics: + - type: accuracy + value: 69.69480519480518 + - type: f1 + value: 68.4951487448326 + - type: f1_weighted + value: 68.4951487448326 + - type: main_score + value: 69.69480519480518 + task: + type: Classification + - dataset: + config: default + name: MTEB BiorxivClusteringP2P (default) + revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40 + split: test + type: mteb/biorxiv-clustering-p2p + metrics: + - type: main_score + value: 29.09840292638104 + - type: v_measure + value: 29.09840292638104 + - type: v_measure_std + value: 0.8130265845858159 + task: + type: Clustering + - dataset: + config: default + name: MTEB BiorxivClusteringS2S (default) + revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908 + split: test + type: mteb/biorxiv-clustering-s2s + metrics: + - type: main_score + value: 18.622216350398414 + - type: v_measure + value: 18.622216350398414 + - type: v_measure_std + value: 0.6293204014994399 + task: + type: Clustering + - dataset: + config: default + name: MTEB CQADupstackAndroidRetrieval (default) + revision: f46a197baaae43b4f621051089b82a364682dfeb + split: test + type: mteb/cqadupstack-android + metrics: + - type: main_score + value: 28.634999999999998 + - type: map_at_1 + value: 17.152 + - type: map_at_10 + value: 23.895 + - type: map_at_100 + value: 25.091 + - type: map_at_1000 + value: 25.25 + - type: map_at_20 + value: 24.572 + - type: map_at_3 + value: 21.568 + - type: map_at_5 + value: 22.704 + - type: mrr_at_1 + value: 22.317596566523605 + - type: mrr_at_10 + value: 29.185627994640857 + - type: mrr_at_100 + value: 30.128435629032374 + - type: mrr_at_1000 + value: 30.21421740576337 + - type: mrr_at_20 + value: 29.74474169170152 + - type: mrr_at_3 + value: 27.014783023366718 + - type: mrr_at_5 + value: 28.130662851692893 + - type: nauc_map_at_1000_diff1 + value: 45.49079878739815 + - type: nauc_map_at_1000_max + value: 29.534546072029517 + - type: nauc_map_at_1000_std + value: -0.5089249190300318 + - type: nauc_map_at_100_diff1 + value: 45.49759783576997 + - type: nauc_map_at_100_max + value: 29.544604885435792 + - type: nauc_map_at_100_std + value: -0.5179655931207162 + - type: nauc_map_at_10_diff1 + value: 45.976510563301666 + - type: nauc_map_at_10_max + value: 29.21402423851111 + - type: nauc_map_at_10_std + value: -1.0085014148488038 + - type: nauc_map_at_1_diff1 + value: 52.75741998819541 + - type: nauc_map_at_1_max + value: 29.368585078235753 + - type: nauc_map_at_1_std + value: -3.5815403185371486 + - type: nauc_map_at_20_diff1 + value: 45.506002175670076 + - type: nauc_map_at_20_max + value: 29.423435660254604 + - type: nauc_map_at_20_std + value: -0.9287391587149738 + - type: nauc_map_at_3_diff1 + value: 47.81373253580316 + - type: nauc_map_at_3_max + value: 29.081148421629916 + - type: nauc_map_at_3_std + value: -2.7629811812032585 + - type: nauc_map_at_5_diff1 + value: 46.55185813784054 + - type: nauc_map_at_5_max + value: 28.961901421879467 + - type: nauc_map_at_5_std + value: -1.937601506223827 + - type: nauc_mrr_at_1000_diff1 + value: 43.614839323060764 + - type: nauc_mrr_at_1000_max + value: 30.960425002935082 + - type: nauc_mrr_at_1000_std + value: -0.5398301648022389 + - type: nauc_mrr_at_100_diff1 + value: 43.59654297191633 + - type: nauc_mrr_at_100_max + value: 30.94533105496734 + - type: nauc_mrr_at_100_std + value: -0.49784542963130296 + - type: nauc_mrr_at_10_diff1 + value: 43.81685974405299 + - type: nauc_mrr_at_10_max + value: 31.028568038527926 + - type: nauc_mrr_at_10_std + value: -0.8292434904346165 + - type: nauc_mrr_at_1_diff1 + value: 49.9370433271119 + - type: nauc_mrr_at_1_max + value: 33.134285195236465 + - type: nauc_mrr_at_1_std + value: -2.7356567659695012 + - type: nauc_mrr_at_20_diff1 + value: 43.584578326941525 + - type: nauc_mrr_at_20_max + value: 31.0127925760322 + - type: nauc_mrr_at_20_std + value: -0.8000020821062815 + - type: nauc_mrr_at_3_diff1 + value: 45.11476358733441 + - type: nauc_mrr_at_3_max + value: 31.409901055954027 + - type: nauc_mrr_at_3_std + value: -2.676967528841609 + - type: nauc_mrr_at_5_diff1 + value: 44.16805851271991 + - type: nauc_mrr_at_5_max + value: 30.88527971859746 + - type: nauc_mrr_at_5_std + value: -1.726462045900353 + - type: nauc_ndcg_at_1000_diff1 + value: 41.77717839292474 + - type: nauc_ndcg_at_1000_max + value: 29.996487458093927 + - type: nauc_ndcg_at_1000_std + value: 3.8581103831063466 + - type: nauc_ndcg_at_100_diff1 + value: 41.54727934976656 + - type: nauc_ndcg_at_100_max + value: 29.71085105251084 + - type: nauc_ndcg_at_100_std + value: 4.118793757322192 + - type: nauc_ndcg_at_10_diff1 + value: 42.71626918562738 + - type: nauc_ndcg_at_10_max + value: 29.12514000395598 + - type: nauc_ndcg_at_10_std + value: 0.9327538498100547 + - type: nauc_ndcg_at_1_diff1 + value: 49.9370433271119 + - type: nauc_ndcg_at_1_max + value: 33.134285195236465 + - type: nauc_ndcg_at_1_std + value: -2.7356567659695012 + - type: nauc_ndcg_at_20_diff1 + value: 41.71058666775361 + - type: nauc_ndcg_at_20_max + value: 29.2397176902118 + - type: nauc_ndcg_at_20_std + value: 1.1399320166488158 + - type: nauc_ndcg_at_3_diff1 + value: 44.328171353087356 + - type: nauc_ndcg_at_3_max + value: 29.225950057257126 + - type: nauc_ndcg_at_3_std + value: -1.9391744610181159 + - type: nauc_ndcg_at_5_diff1 + value: 43.074003557819154 + - type: nauc_ndcg_at_5_max + value: 28.686726626031327 + - type: nauc_ndcg_at_5_std + value: -0.9025430391878352 + - type: nauc_precision_at_1000_diff1 + value: 0.019262094978555112 + - type: nauc_precision_at_1000_max + value: 1.4280147129801406 + - type: nauc_precision_at_1000_std + value: 5.237394137713056 + - type: nauc_precision_at_100_diff1 + value: 9.734198894095327 + - type: nauc_precision_at_100_max + value: 17.994270805040287 + - type: nauc_precision_at_100_std + value: 10.816224870855196 + - type: nauc_precision_at_10_diff1 + value: 24.273335995399528 + - type: nauc_precision_at_10_max + value: 25.012150937872573 + - type: nauc_precision_at_10_std + value: 5.706335599631678 + - type: nauc_precision_at_1_diff1 + value: 49.9370433271119 + - type: nauc_precision_at_1_max + value: 33.134285195236465 + - type: nauc_precision_at_1_std + value: -2.7356567659695012 + - type: nauc_precision_at_20_diff1 + value: 18.199627802448852 + - type: nauc_precision_at_20_max + value: 24.55969777615955 + - type: nauc_precision_at_20_std + value: 5.528931520512831 + - type: nauc_precision_at_3_diff1 + value: 35.21809033465108 + - type: nauc_precision_at_3_max + value: 28.81285998763693 + - type: nauc_precision_at_3_std + value: -1.3820729819891873 + - type: nauc_precision_at_5_diff1 + value: 28.964838568128283 + - type: nauc_precision_at_5_max + value: 27.919889820702092 + - type: nauc_precision_at_5_std + value: 0.8326959136541273 + - type: nauc_recall_at_1000_diff1 + value: 21.23258255267272 + - type: nauc_recall_at_1000_max + value: 30.758022399262604 + - type: nauc_recall_at_1000_std + value: 40.400857317855966 + - type: nauc_recall_at_100_diff1 + value: 27.045466279262133 + - type: nauc_recall_at_100_max + value: 26.471211331500676 + - type: nauc_recall_at_100_std + value: 22.702444466758195 + - type: nauc_recall_at_10_diff1 + value: 33.65302461303574 + - type: nauc_recall_at_10_max + value: 25.612926968693827 + - type: nauc_recall_at_10_std + value: 6.310778309941729 + - type: nauc_recall_at_1_diff1 + value: 52.75741998819541 + - type: nauc_recall_at_1_max + value: 29.368585078235753 + - type: nauc_recall_at_1_std + value: -3.5815403185371486 + - type: nauc_recall_at_20_diff1 + value: 29.16482990269768 + - type: nauc_recall_at_20_max + value: 24.902713129283242 + - type: nauc_recall_at_20_std + value: 6.842748407870703 + - type: nauc_recall_at_3_diff1 + value: 40.48285894994969 + - type: nauc_recall_at_3_max + value: 25.25195012757927 + - type: nauc_recall_at_3_std + value: -1.923836712152538 + - type: nauc_recall_at_5_diff1 + value: 35.61485776280015 + - type: nauc_recall_at_5_max + value: 23.60803593241084 + - type: nauc_recall_at_5_std + value: 1.2754180362185552 + - type: ndcg_at_1 + value: 22.317999999999998 + - type: ndcg_at_10 + value: 28.634999999999998 + - type: ndcg_at_100 + value: 33.954 + - type: ndcg_at_1000 + value: 37.467 + - type: ndcg_at_20 + value: 30.75 + - type: ndcg_at_3 + value: 24.956 + - type: ndcg_at_5 + value: 26.308 + - type: precision_at_1 + value: 22.317999999999998 + - type: precision_at_10 + value: 5.651 + - type: precision_at_100 + value: 1.039 + - type: precision_at_1000 + value: 0.166 + - type: precision_at_20 + value: 3.4979999999999998 + - type: precision_at_3 + value: 12.256 + - type: precision_at_5 + value: 8.755 + - type: recall_at_1 + value: 17.152 + - type: recall_at_10 + value: 37.447 + - type: recall_at_100 + value: 60.699000000000005 + - type: recall_at_1000 + value: 85.014 + - type: recall_at_20 + value: 45.405 + - type: recall_at_3 + value: 26.143 + - type: recall_at_5 + value: 30.441000000000003 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackEnglishRetrieval (default) + revision: ad9991cb51e31e31e430383c75ffb2885547b5f0 + split: test + type: mteb/cqadupstack-english + metrics: + - type: main_score + value: 23.933 + - type: map_at_1 + value: 15.534999999999998 + - type: map_at_10 + value: 20.494 + - type: map_at_100 + value: 21.234 + - type: map_at_1000 + value: 21.346999999999998 + - type: map_at_20 + value: 20.842 + - type: map_at_3 + value: 18.716 + - type: map_at_5 + value: 19.694 + - type: mrr_at_1 + value: 18.85350318471338 + - type: mrr_at_10 + value: 23.976417955717306 + - type: mrr_at_100 + value: 24.626250491278405 + - type: mrr_at_1000 + value: 24.699274897180697 + - type: mrr_at_20 + value: 24.31440451029886 + - type: mrr_at_3 + value: 22.218683651804664 + - type: mrr_at_5 + value: 23.209129511677258 + - type: nauc_map_at_1000_diff1 + value: 37.74002837588212 + - type: nauc_map_at_1000_max + value: 11.52596069444164 + - type: nauc_map_at_1000_std + value: 1.5396386499345946 + - type: nauc_map_at_100_diff1 + value: 37.769383414720515 + - type: nauc_map_at_100_max + value: 11.472584783973845 + - type: nauc_map_at_100_std + value: 1.4530341264663071 + - type: nauc_map_at_10_diff1 + value: 38.14778408943225 + - type: nauc_map_at_10_max + value: 11.457324548618868 + - type: nauc_map_at_10_std + value: 0.8673572005420617 + - type: nauc_map_at_1_diff1 + value: 45.248945131354226 + - type: nauc_map_at_1_max + value: 13.305886052642421 + - type: nauc_map_at_1_std + value: -0.8554239133186292 + - type: nauc_map_at_20_diff1 + value: 37.94962093945997 + - type: nauc_map_at_20_max + value: 11.368934419724535 + - type: nauc_map_at_20_std + value: 1.176488837859526 + - type: nauc_map_at_3_diff1 + value: 38.816860264330906 + - type: nauc_map_at_3_max + value: 12.244663804798567 + - type: nauc_map_at_3_std + value: -0.7044706920393753 + - type: nauc_map_at_5_diff1 + value: 38.20664251362074 + - type: nauc_map_at_5_max + value: 11.58930422846952 + - type: nauc_map_at_5_std + value: 0.24788040735166877 + - type: nauc_mrr_at_1000_diff1 + value: 36.01366865176011 + - type: nauc_mrr_at_1000_max + value: 12.692732314206609 + - type: nauc_mrr_at_1000_std + value: 3.24063469677308 + - type: nauc_mrr_at_100_diff1 + value: 36.01759994326736 + - type: nauc_mrr_at_100_max + value: 12.688310838686673 + - type: nauc_mrr_at_100_std + value: 3.243753049954808 + - type: nauc_mrr_at_10_diff1 + value: 36.27391277272198 + - type: nauc_mrr_at_10_max + value: 12.668653023640713 + - type: nauc_mrr_at_10_std + value: 2.9800614398297265 + - type: nauc_mrr_at_1_diff1 + value: 42.03008021102266 + - type: nauc_mrr_at_1_max + value: 15.464219677345653 + - type: nauc_mrr_at_1_std + value: 1.9384689651010967 + - type: nauc_mrr_at_20_diff1 + value: 36.081487674680965 + - type: nauc_mrr_at_20_max + value: 12.63192966261147 + - type: nauc_mrr_at_20_std + value: 3.1357738226142096 + - type: nauc_mrr_at_3_diff1 + value: 36.74126358143279 + - type: nauc_mrr_at_3_max + value: 13.486458067334103 + - type: nauc_mrr_at_3_std + value: 2.0734773435525886 + - type: nauc_mrr_at_5_diff1 + value: 36.21555621305363 + - type: nauc_mrr_at_5_max + value: 12.776138636129582 + - type: nauc_mrr_at_5_std + value: 2.7040718821656418 + - type: nauc_ndcg_at_1000_diff1 + value: 34.17368469001136 + - type: nauc_ndcg_at_1000_max + value: 11.048310100081675 + - type: nauc_ndcg_at_1000_std + value: 5.07898838985954 + - type: nauc_ndcg_at_100_diff1 + value: 34.55833751183066 + - type: nauc_ndcg_at_100_max + value: 10.558354467049766 + - type: nauc_ndcg_at_100_std + value: 4.549167498357635 + - type: nauc_ndcg_at_10_diff1 + value: 35.792555836995334 + - type: nauc_ndcg_at_10_max + value: 10.462185132989973 + - type: nauc_ndcg_at_10_std + value: 2.8477903406821183 + - type: nauc_ndcg_at_1_diff1 + value: 42.03008021102266 + - type: nauc_ndcg_at_1_max + value: 15.464219677345653 + - type: nauc_ndcg_at_1_std + value: 1.9384689651010967 + - type: nauc_ndcg_at_20_diff1 + value: 35.26991472528924 + - type: nauc_ndcg_at_20_max + value: 10.069855031533036 + - type: nauc_ndcg_at_20_std + value: 3.376655328346143 + - type: nauc_ndcg_at_3_diff1 + value: 36.37053997114841 + - type: nauc_ndcg_at_3_max + value: 12.224505873247024 + - type: nauc_ndcg_at_3_std + value: 0.6624194427639196 + - type: nauc_ndcg_at_5_diff1 + value: 35.70090332699717 + - type: nauc_ndcg_at_5_max + value: 10.718660748235513 + - type: nauc_ndcg_at_5_std + value: 1.8377187458851245 + - type: nauc_precision_at_1000_diff1 + value: -4.890564070316116 + - type: nauc_precision_at_1000_max + value: 8.308792103010054 + - type: nauc_precision_at_1000_std + value: 20.927425655021437 + - type: nauc_precision_at_100_diff1 + value: 6.781480258638198 + - type: nauc_precision_at_100_max + value: 9.14338916106647 + - type: nauc_precision_at_100_std + value: 19.5689773619807 + - type: nauc_precision_at_10_diff1 + value: 22.120003359351454 + - type: nauc_precision_at_10_max + value: 9.880724079161961 + - type: nauc_precision_at_10_std + value: 11.898292477010239 + - type: nauc_precision_at_1_diff1 + value: 42.03008021102266 + - type: nauc_precision_at_1_max + value: 15.464219677345653 + - type: nauc_precision_at_1_std + value: 1.9384689651010967 + - type: nauc_precision_at_20_diff1 + value: 17.586401294319412 + - type: nauc_precision_at_20_max + value: 8.286556086704074 + - type: nauc_precision_at_20_std + value: 14.20376727306384 + - type: nauc_precision_at_3_diff1 + value: 27.877469254113386 + - type: nauc_precision_at_3_max + value: 12.208775338231302 + - type: nauc_precision_at_3_std + value: 3.874876685435265 + - type: nauc_precision_at_5_diff1 + value: 24.786024793792617 + - type: nauc_precision_at_5_max + value: 10.355536887167844 + - type: nauc_precision_at_5_std + value: 8.000371130105563 + - type: nauc_recall_at_1000_diff1 + value: 21.767117265109228 + - type: nauc_recall_at_1000_max + value: 7.388142374547023 + - type: nauc_recall_at_1000_std + value: 13.274812043748158 + - type: nauc_recall_at_100_diff1 + value: 25.468443648444573 + - type: nauc_recall_at_100_max + value: 6.331483861859732 + - type: nauc_recall_at_100_std + value: 10.596959963008198 + - type: nauc_recall_at_10_diff1 + value: 30.928988191036368 + - type: nauc_recall_at_10_max + value: 6.321847209343759 + - type: nauc_recall_at_10_std + value: 4.4911642118461685 + - type: nauc_recall_at_1_diff1 + value: 45.248945131354226 + - type: nauc_recall_at_1_max + value: 13.305886052642421 + - type: nauc_recall_at_1_std + value: -0.8554239133186292 + - type: nauc_recall_at_20_diff1 + value: 28.79337381513034 + - type: nauc_recall_at_20_max + value: 5.113928725475475 + - type: nauc_recall_at_20_std + value: 6.184413133197319 + - type: nauc_recall_at_3_diff1 + value: 32.39614504895464 + - type: nauc_recall_at_3_max + value: 9.328104228537699 + - type: nauc_recall_at_3_std + value: -0.5296736724789125 + - type: nauc_recall_at_5_diff1 + value: 31.294125355167253 + - type: nauc_recall_at_5_max + value: 6.491641210370972 + - type: nauc_recall_at_5_std + value: 2.040846304053944 + - type: ndcg_at_1 + value: 18.854000000000003 + - type: ndcg_at_10 + value: 23.933 + - type: ndcg_at_100 + value: 27.593 + - type: ndcg_at_1000 + value: 30.320000000000004 + - type: ndcg_at_20 + value: 25.068 + - type: ndcg_at_3 + value: 20.806 + - type: ndcg_at_5 + value: 22.216 + - type: precision_at_1 + value: 18.854000000000003 + - type: precision_at_10 + value: 4.312 + - type: precision_at_100 + value: 0.773 + - type: precision_at_1000 + value: 0.124 + - type: precision_at_20 + value: 2.557 + - type: precision_at_3 + value: 9.575 + - type: precision_at_5 + value: 6.93 + - type: recall_at_1 + value: 15.534999999999998 + - type: recall_at_10 + value: 30.865 + - type: recall_at_100 + value: 47.044999999999995 + - type: recall_at_1000 + value: 65.706 + - type: recall_at_20 + value: 35.043 + - type: recall_at_3 + value: 22.034000000000002 + - type: recall_at_5 + value: 25.663000000000004 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackGamingRetrieval (default) + revision: 4885aa143210c98657558c04aaf3dc47cfb54340 + split: test + type: mteb/cqadupstack-gaming + metrics: + - type: main_score + value: 35.807 + - type: map_at_1 + value: 22.628999999999998 + - type: map_at_10 + value: 31.06 + - type: map_at_100 + value: 32.124 + - type: map_at_1000 + value: 32.228 + - type: map_at_20 + value: 31.714 + - type: map_at_3 + value: 28.576 + - type: map_at_5 + value: 30.049999999999997 + - type: mrr_at_1 + value: 26.332288401253916 + - type: mrr_at_10 + value: 34.26966711449467 + - type: mrr_at_100 + value: 35.10307061641104 + - type: mrr_at_1000 + value: 35.175621663030384 + - type: mrr_at_20 + value: 34.77207492905975 + - type: mrr_at_3 + value: 31.985370950888154 + - type: mrr_at_5 + value: 33.38035527690695 + - type: nauc_map_at_1000_diff1 + value: 41.26538406776667 + - type: nauc_map_at_1000_max + value: 22.089808526173663 + - type: nauc_map_at_1000_std + value: -6.82680457634158 + - type: nauc_map_at_100_diff1 + value: 41.272836431982014 + - type: nauc_map_at_100_max + value: 22.06375392023965 + - type: nauc_map_at_100_std + value: -6.876803386437694 + - type: nauc_map_at_10_diff1 + value: 41.427308262702496 + - type: nauc_map_at_10_max + value: 21.774850849917172 + - type: nauc_map_at_10_std + value: -7.521250762475834 + - type: nauc_map_at_1_diff1 + value: 47.0682194607475 + - type: nauc_map_at_1_max + value: 19.7271244754008 + - type: nauc_map_at_1_std + value: -8.387247040771014 + - type: nauc_map_at_20_diff1 + value: 41.250225334717 + - type: nauc_map_at_20_max + value: 21.952825244751704 + - type: nauc_map_at_20_std + value: -7.089842060679384 + - type: nauc_map_at_3_diff1 + value: 42.11071549668694 + - type: nauc_map_at_3_max + value: 20.334031983597875 + - type: nauc_map_at_3_std + value: -8.655163177779556 + - type: nauc_map_at_5_diff1 + value: 41.55276805478624 + - type: nauc_map_at_5_max + value: 21.341662053429037 + - type: nauc_map_at_5_std + value: -8.252224038913697 + - type: nauc_mrr_at_1000_diff1 + value: 41.1807053561387 + - type: nauc_mrr_at_1000_max + value: 24.141034562422607 + - type: nauc_mrr_at_1000_std + value: -4.582768753751603 + - type: nauc_mrr_at_100_diff1 + value: 41.17248963594074 + - type: nauc_mrr_at_100_max + value: 24.14923648452245 + - type: nauc_mrr_at_100_std + value: -4.586237441845526 + - type: nauc_mrr_at_10_diff1 + value: 41.29997961211437 + - type: nauc_mrr_at_10_max + value: 24.18197800438829 + - type: nauc_mrr_at_10_std + value: -4.92059802789352 + - type: nauc_mrr_at_1_diff1 + value: 47.22202156377031 + - type: nauc_mrr_at_1_max + value: 22.81382395299871 + - type: nauc_mrr_at_1_std + value: -6.511731680897033 + - type: nauc_mrr_at_20_diff1 + value: 41.12982414886362 + - type: nauc_mrr_at_20_max + value: 24.090339351286172 + - type: nauc_mrr_at_20_std + value: -4.714787687162255 + - type: nauc_mrr_at_3_diff1 + value: 42.03685291338169 + - type: nauc_mrr_at_3_max + value: 23.46979265259633 + - type: nauc_mrr_at_3_std + value: -5.746637512261208 + - type: nauc_mrr_at_5_diff1 + value: 41.41305039216194 + - type: nauc_mrr_at_5_max + value: 24.000287116591235 + - type: nauc_mrr_at_5_std + value: -5.470798244701126 + - type: nauc_ndcg_at_1000_diff1 + value: 38.894631119522074 + - type: nauc_ndcg_at_1000_max + value: 24.067285655689073 + - type: nauc_ndcg_at_1000_std + value: -2.6694327561644458 + - type: nauc_ndcg_at_100_diff1 + value: 38.82244557925009 + - type: nauc_ndcg_at_100_max + value: 23.997724699889826 + - type: nauc_ndcg_at_100_std + value: -3.0442083060901655 + - type: nauc_ndcg_at_10_diff1 + value: 39.35732365524711 + - type: nauc_ndcg_at_10_max + value: 23.26445091825662 + - type: nauc_ndcg_at_10_std + value: -5.702352367922923 + - type: nauc_ndcg_at_1_diff1 + value: 47.22202156377031 + - type: nauc_ndcg_at_1_max + value: 22.81382395299871 + - type: nauc_ndcg_at_1_std + value: -6.511731680897033 + - type: nauc_ndcg_at_20_diff1 + value: 38.71915510942565 + - type: nauc_ndcg_at_20_max + value: 23.38127406835881 + - type: nauc_ndcg_at_20_std + value: -4.628020864250285 + - type: nauc_ndcg_at_3_diff1 + value: 40.65241506559219 + - type: nauc_ndcg_at_3_max + value: 21.353018199050783 + - type: nauc_ndcg_at_3_std + value: -7.621281922991216 + - type: nauc_ndcg_at_5_diff1 + value: 39.66904523712232 + - type: nauc_ndcg_at_5_max + value: 22.62846235537732 + - type: nauc_ndcg_at_5_std + value: -7.238655338688719 + - type: nauc_precision_at_1000_diff1 + value: -0.10674589877264098 + - type: nauc_precision_at_1000_max + value: 22.880044900789883 + - type: nauc_precision_at_1000_std + value: 25.286841157887324 + - type: nauc_precision_at_100_diff1 + value: 12.494580543979819 + - type: nauc_precision_at_100_max + value: 26.407646479434348 + - type: nauc_precision_at_100_std + value: 18.00786656478982 + - type: nauc_precision_at_10_diff1 + value: 25.425846313220536 + - type: nauc_precision_at_10_max + value: 27.282619860852453 + - type: nauc_precision_at_10_std + value: 3.254416482888394 + - type: nauc_precision_at_1_diff1 + value: 47.22202156377031 + - type: nauc_precision_at_1_max + value: 22.81382395299871 + - type: nauc_precision_at_1_std + value: -6.511731680897033 + - type: nauc_precision_at_20_diff1 + value: 19.841372822387022 + - type: nauc_precision_at_20_max + value: 26.062138234488096 + - type: nauc_precision_at_20_std + value: 8.170520123604948 + - type: nauc_precision_at_3_diff1 + value: 33.490720368183055 + - type: nauc_precision_at_3_max + value: 23.698874436209138 + - type: nauc_precision_at_3_std + value: -4.247457542736893 + - type: nauc_precision_at_5_diff1 + value: 29.5800651053398 + - type: nauc_precision_at_5_max + value: 26.96711574015605 + - type: nauc_precision_at_5_std + value: -1.9260500005639964 + - type: nauc_recall_at_1000_diff1 + value: 21.07089430236666 + - type: nauc_recall_at_1000_max + value: 29.068557444348208 + - type: nauc_recall_at_1000_std + value: 29.467514748549682 + - type: nauc_recall_at_100_diff1 + value: 28.045187225936512 + - type: nauc_recall_at_100_max + value: 27.307019737570975 + - type: nauc_recall_at_100_std + value: 11.826790191487909 + - type: nauc_recall_at_10_diff1 + value: 32.50271652049762 + - type: nauc_recall_at_10_max + value: 23.973682202148833 + - type: nauc_recall_at_10_std + value: -2.840362990837493 + - type: nauc_recall_at_1_diff1 + value: 47.0682194607475 + - type: nauc_recall_at_1_max + value: 19.7271244754008 + - type: nauc_recall_at_1_std + value: -8.387247040771014 + - type: nauc_recall_at_20_diff1 + value: 29.540137183084482 + - type: nauc_recall_at_20_max + value: 24.15868254359213 + - type: nauc_recall_at_20_std + value: 1.1030187029057643 + - type: nauc_recall_at_3_diff1 + value: 36.11833942271728 + - type: nauc_recall_at_3_max + value: 19.70224872745943 + - type: nauc_recall_at_3_std + value: -8.193955960840473 + - type: nauc_recall_at_5_diff1 + value: 33.6864411204656 + - type: nauc_recall_at_5_max + value: 22.29534359351608 + - type: nauc_recall_at_5_std + value: -7.198005144877803 + - type: ndcg_at_1 + value: 26.332 + - type: ndcg_at_10 + value: 35.807 + - type: ndcg_at_100 + value: 40.654 + - type: ndcg_at_1000 + value: 43.19 + - type: ndcg_at_20 + value: 37.854 + - type: ndcg_at_3 + value: 31.238 + - type: ndcg_at_5 + value: 33.6 + - type: precision_at_1 + value: 26.332 + - type: precision_at_10 + value: 6.006 + - type: precision_at_100 + value: 0.922 + - type: precision_at_1000 + value: 0.122 + - type: precision_at_20 + value: 3.5520000000000005 + - type: precision_at_3 + value: 14.19 + - type: precision_at_5 + value: 10.132 + - type: recall_at_1 + value: 22.628999999999998 + - type: recall_at_10 + value: 46.965 + - type: recall_at_100 + value: 68.748 + - type: recall_at_1000 + value: 87.428 + - type: recall_at_20 + value: 54.559999999999995 + - type: recall_at_3 + value: 34.777 + - type: recall_at_5 + value: 40.527 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackGisRetrieval (default) + revision: 5003b3064772da1887988e05400cf3806fe491f2 + split: test + type: mteb/cqadupstack-gis + metrics: + - type: main_score + value: 17.629 + - type: map_at_1 + value: 10.38 + - type: map_at_10 + value: 14.754000000000001 + - type: map_at_100 + value: 15.426 + - type: map_at_1000 + value: 15.534999999999998 + - type: map_at_20 + value: 15.088 + - type: map_at_3 + value: 13.094 + - type: map_at_5 + value: 14.05 + - type: mrr_at_1 + value: 11.299435028248588 + - type: mrr_at_10 + value: 15.835978835978842 + - type: mrr_at_100 + value: 16.534237809945086 + - type: mrr_at_1000 + value: 16.63481660848549 + - type: mrr_at_20 + value: 16.204438309373508 + - type: mrr_at_3 + value: 14.105461393596983 + - type: mrr_at_5 + value: 15.133709981167609 + - type: nauc_map_at_1000_diff1 + value: 30.813780800275577 + - type: nauc_map_at_1000_max + value: 11.782343656250962 + - type: nauc_map_at_1000_std + value: -8.450259569210562 + - type: nauc_map_at_100_diff1 + value: 30.806060913087485 + - type: nauc_map_at_100_max + value: 11.766956595713632 + - type: nauc_map_at_100_std + value: -8.490982686334714 + - type: nauc_map_at_10_diff1 + value: 31.036790223331284 + - type: nauc_map_at_10_max + value: 11.65928648411015 + - type: nauc_map_at_10_std + value: -8.891015250999633 + - type: nauc_map_at_1_diff1 + value: 40.392118725694296 + - type: nauc_map_at_1_max + value: 12.989514645776437 + - type: nauc_map_at_1_std + value: -12.754693269346662 + - type: nauc_map_at_20_diff1 + value: 30.901613440113913 + - type: nauc_map_at_20_max + value: 11.746782149315164 + - type: nauc_map_at_20_std + value: -8.736689401578923 + - type: nauc_map_at_3_diff1 + value: 33.20492753741601 + - type: nauc_map_at_3_max + value: 12.357131975441227 + - type: nauc_map_at_3_std + value: -10.563694590125545 + - type: nauc_map_at_5_diff1 + value: 32.061276704672686 + - type: nauc_map_at_5_max + value: 12.065660289608578 + - type: nauc_map_at_5_std + value: -9.627361978016017 + - type: nauc_mrr_at_1000_diff1 + value: 29.18067582766131 + - type: nauc_mrr_at_1000_max + value: 12.945793155078967 + - type: nauc_mrr_at_1000_std + value: -7.3265109617304685 + - type: nauc_mrr_at_100_diff1 + value: 29.163181593347904 + - type: nauc_mrr_at_100_max + value: 12.948132882807926 + - type: nauc_mrr_at_100_std + value: -7.351935545294857 + - type: nauc_mrr_at_10_diff1 + value: 29.31938739231278 + - type: nauc_mrr_at_10_max + value: 12.806414703000188 + - type: nauc_mrr_at_10_std + value: -7.610287140903537 + - type: nauc_mrr_at_1_diff1 + value: 38.89154503294042 + - type: nauc_mrr_at_1_max + value: 14.139403084242247 + - type: nauc_mrr_at_1_std + value: -11.312198001697299 + - type: nauc_mrr_at_20_diff1 + value: 29.194444050920183 + - type: nauc_mrr_at_20_max + value: 12.904450862547447 + - type: nauc_mrr_at_20_std + value: -7.545415671110467 + - type: nauc_mrr_at_3_diff1 + value: 31.393025095982384 + - type: nauc_mrr_at_3_max + value: 13.446210300659404 + - type: nauc_mrr_at_3_std + value: -8.842651857422517 + - type: nauc_mrr_at_5_diff1 + value: 30.22907968906219 + - type: nauc_mrr_at_5_max + value: 13.073366417712586 + - type: nauc_mrr_at_5_std + value: -8.226431830427765 + - type: nauc_ndcg_at_1000_diff1 + value: 26.264947886953237 + - type: nauc_ndcg_at_1000_max + value: 11.441384810090252 + - type: nauc_ndcg_at_1000_std + value: -4.395454030779977 + - type: nauc_ndcg_at_100_diff1 + value: 26.0471151899472 + - type: nauc_ndcg_at_100_max + value: 11.093079614438034 + - type: nauc_ndcg_at_100_std + value: -4.9764023249186655 + - type: nauc_ndcg_at_10_diff1 + value: 26.90251142153969 + - type: nauc_ndcg_at_10_max + value: 10.97337729814191 + - type: nauc_ndcg_at_10_std + value: -6.705324064661864 + - type: nauc_ndcg_at_1_diff1 + value: 38.89154503294042 + - type: nauc_ndcg_at_1_max + value: 14.139403084242247 + - type: nauc_ndcg_at_1_std + value: -11.312198001697299 + - type: nauc_ndcg_at_20_diff1 + value: 26.465932737299585 + - type: nauc_ndcg_at_20_max + value: 11.089006750559657 + - type: nauc_ndcg_at_20_std + value: -6.36961075704304 + - type: nauc_ndcg_at_3_diff1 + value: 30.565609563262058 + - type: nauc_ndcg_at_3_max + value: 12.086090885642179 + - type: nauc_ndcg_at_3_std + value: -9.76046084488079 + - type: nauc_ndcg_at_5_diff1 + value: 28.901329485568876 + - type: nauc_ndcg_at_5_max + value: 11.755991552389204 + - type: nauc_ndcg_at_5_std + value: -8.214029330866332 + - type: nauc_precision_at_1000_diff1 + value: 5.893427497520307 + - type: nauc_precision_at_1000_max + value: 13.962546949241588 + - type: nauc_precision_at_1000_std + value: 11.182186647992918 + - type: nauc_precision_at_100_diff1 + value: 14.122452287718449 + - type: nauc_precision_at_100_max + value: 11.56106074944499 + - type: nauc_precision_at_100_std + value: 5.394836213772653 + - type: nauc_precision_at_10_diff1 + value: 17.44514141046529 + - type: nauc_precision_at_10_max + value: 10.595913542289864 + - type: nauc_precision_at_10_std + value: -0.20582788920299083 + - type: nauc_precision_at_1_diff1 + value: 38.89154503294042 + - type: nauc_precision_at_1_max + value: 14.139403084242247 + - type: nauc_precision_at_1_std + value: -11.312198001697299 + - type: nauc_precision_at_20_diff1 + value: 16.858817787085904 + - type: nauc_precision_at_20_max + value: 10.797716199244915 + - type: nauc_precision_at_20_std + value: -0.10821855328410326 + - type: nauc_precision_at_3_diff1 + value: 24.700655044693608 + - type: nauc_precision_at_3_max + value: 13.11860977106586 + - type: nauc_precision_at_3_std + value: -7.129190710723354 + - type: nauc_precision_at_5_diff1 + value: 21.49719563326281 + - type: nauc_precision_at_5_max + value: 12.493693151305854 + - type: nauc_precision_at_5_std + value: -4.332210627942629 + - type: nauc_recall_at_1000_diff1 + value: 14.925915862719302 + - type: nauc_recall_at_1000_max + value: 8.497504317724683 + - type: nauc_recall_at_1000_std + value: 8.781321940759797 + - type: nauc_recall_at_100_diff1 + value: 15.908339423365408 + - type: nauc_recall_at_100_max + value: 7.574805370380677 + - type: nauc_recall_at_100_std + value: 2.9546971482740427 + - type: nauc_recall_at_10_diff1 + value: 18.479088740671163 + - type: nauc_recall_at_10_max + value: 8.196706188245905 + - type: nauc_recall_at_10_std + value: -2.6997578673469107 + - type: nauc_recall_at_1_diff1 + value: 40.392118725694296 + - type: nauc_recall_at_1_max + value: 12.989514645776437 + - type: nauc_recall_at_1_std + value: -12.754693269346662 + - type: nauc_recall_at_20_diff1 + value: 17.355050199783538 + - type: nauc_recall_at_20_max + value: 8.24865474928656 + - type: nauc_recall_at_20_std + value: -1.9444165911432059 + - type: nauc_recall_at_3_diff1 + value: 25.991866279577998 + - type: nauc_recall_at_3_max + value: 10.780718288384648 + - type: nauc_recall_at_3_std + value: -9.020518000524683 + - type: nauc_recall_at_5_diff1 + value: 22.825582625970096 + - type: nauc_recall_at_5_max + value: 9.895504719395994 + - type: nauc_recall_at_5_std + value: -5.958812555760698 + - type: ndcg_at_1 + value: 11.299 + - type: ndcg_at_10 + value: 17.629 + - type: ndcg_at_100 + value: 21.437 + - type: ndcg_at_1000 + value: 24.599 + - type: ndcg_at_20 + value: 18.864 + - type: ndcg_at_3 + value: 14.221 + - type: ndcg_at_5 + value: 15.950000000000001 + - type: precision_at_1 + value: 11.299 + - type: precision_at_10 + value: 2.904 + - type: precision_at_100 + value: 0.515 + - type: precision_at_1000 + value: 0.082 + - type: precision_at_20 + value: 1.7340000000000002 + - type: precision_at_3 + value: 6.064 + - type: precision_at_5 + value: 4.61 + - type: recall_at_1 + value: 10.38 + - type: recall_at_10 + value: 25.627 + - type: recall_at_100 + value: 43.95 + - type: recall_at_1000 + value: 68.49 + - type: recall_at_20 + value: 30.320000000000004 + - type: recall_at_3 + value: 16.329 + - type: recall_at_5 + value: 20.561 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackMathematicaRetrieval (default) + revision: 90fceea13679c63fe563ded68f3b6f06e50061de + split: test + type: mteb/cqadupstack-mathematica + metrics: + - type: main_score + value: 11.477 + - type: map_at_1 + value: 5.978 + - type: map_at_10 + value: 9.094 + - type: map_at_100 + value: 9.876 + - type: map_at_1000 + value: 9.994 + - type: map_at_20 + value: 9.472 + - type: map_at_3 + value: 7.939 + - type: map_at_5 + value: 8.464 + - type: mrr_at_1 + value: 7.213930348258707 + - type: mrr_at_10 + value: 11.07784490247177 + - type: mrr_at_100 + value: 11.899767012693667 + - type: mrr_at_1000 + value: 11.998411801437172 + - type: mrr_at_20 + value: 11.508338702856639 + - type: mrr_at_3 + value: 9.784411276948592 + - type: mrr_at_5 + value: 10.350331674958541 + - type: nauc_map_at_1000_diff1 + value: 13.591085928739844 + - type: nauc_map_at_1000_max + value: 6.290217870776886 + - type: nauc_map_at_1000_std + value: 2.392951276884633 + - type: nauc_map_at_100_diff1 + value: 13.668656542290492 + - type: nauc_map_at_100_max + value: 6.258917946356381 + - type: nauc_map_at_100_std + value: 2.3266445676100136 + - type: nauc_map_at_10_diff1 + value: 13.250096964157141 + - type: nauc_map_at_10_max + value: 5.813606943292423 + - type: nauc_map_at_10_std + value: 1.7271825359327229 + - type: nauc_map_at_1_diff1 + value: 20.218709814462795 + - type: nauc_map_at_1_max + value: 8.036844386597277 + - type: nauc_map_at_1_std + value: 4.620772690437222 + - type: nauc_map_at_20_diff1 + value: 13.21941234439436 + - type: nauc_map_at_20_max + value: 6.397052877775318 + - type: nauc_map_at_20_std + value: 2.0208479181057863 + - type: nauc_map_at_3_diff1 + value: 14.253784181301269 + - type: nauc_map_at_3_max + value: 5.2244549800840705 + - type: nauc_map_at_3_std + value: 3.368522302214036 + - type: nauc_map_at_5_diff1 + value: 13.044211991524158 + - type: nauc_map_at_5_max + value: 4.906068326503013 + - type: nauc_map_at_5_std + value: 1.338022929716299 + - type: nauc_mrr_at_1000_diff1 + value: 12.290215308682262 + - type: nauc_mrr_at_1000_max + value: 7.643442169311612 + - type: nauc_mrr_at_1000_std + value: 4.653330662838167 + - type: nauc_mrr_at_100_diff1 + value: 12.356655427085565 + - type: nauc_mrr_at_100_max + value: 7.628444205245759 + - type: nauc_mrr_at_100_std + value: 4.63571924333444 + - type: nauc_mrr_at_10_diff1 + value: 11.955038091668206 + - type: nauc_mrr_at_10_max + value: 7.3633463435739746 + - type: nauc_mrr_at_10_std + value: 4.340105432925798 + - type: nauc_mrr_at_1_diff1 + value: 18.54493340747909 + - type: nauc_mrr_at_1_max + value: 8.77355237570233 + - type: nauc_mrr_at_1_std + value: 5.594302960623576 + - type: nauc_mrr_at_20_diff1 + value: 12.046481294394413 + - type: nauc_mrr_at_20_max + value: 7.690200754146728 + - type: nauc_mrr_at_20_std + value: 4.4655331328551195 + - type: nauc_mrr_at_3_diff1 + value: 12.435651201239532 + - type: nauc_mrr_at_3_max + value: 6.558488098778106 + - type: nauc_mrr_at_3_std + value: 5.698098301646967 + - type: nauc_mrr_at_5_diff1 + value: 11.748052022736733 + - type: nauc_mrr_at_5_max + value: 6.5271856425922365 + - type: nauc_mrr_at_5_std + value: 4.145794538156133 + - type: nauc_ndcg_at_1000_diff1 + value: 13.288712098878081 + - type: nauc_ndcg_at_1000_max + value: 7.657748149258937 + - type: nauc_ndcg_at_1000_std + value: 5.545508293121399 + - type: nauc_ndcg_at_100_diff1 + value: 14.885215517504447 + - type: nauc_ndcg_at_100_max + value: 7.2418991240971735 + - type: nauc_ndcg_at_100_std + value: 3.966737924400339 + - type: nauc_ndcg_at_10_diff1 + value: 11.727912572806847 + - type: nauc_ndcg_at_10_max + value: 6.433084111905497 + - type: nauc_ndcg_at_10_std + value: 1.774941654953616 + - type: nauc_ndcg_at_1_diff1 + value: 18.54493340747909 + - type: nauc_ndcg_at_1_max + value: 8.77355237570233 + - type: nauc_ndcg_at_1_std + value: 5.594302960623576 + - type: nauc_ndcg_at_20_diff1 + value: 11.790912817274762 + - type: nauc_ndcg_at_20_max + value: 7.776091543736068 + - type: nauc_ndcg_at_20_std + value: 2.6016157495554073 + - type: nauc_ndcg_at_3_diff1 + value: 12.439122277051485 + - type: nauc_ndcg_at_3_max + value: 5.174924004324122 + - type: nauc_ndcg_at_3_std + value: 3.860294561139816 + - type: nauc_ndcg_at_5_diff1 + value: 10.913762290310355 + - type: nauc_ndcg_at_5_max + value: 4.575898104548388 + - type: nauc_ndcg_at_5_std + value: 0.8550022603482168 + - type: nauc_precision_at_1000_diff1 + value: 0.8641627546620656 + - type: nauc_precision_at_1000_max + value: 7.136167893836067 + - type: nauc_precision_at_1000_std + value: 7.008996062106819 + - type: nauc_precision_at_100_diff1 + value: 13.0673021579007 + - type: nauc_precision_at_100_max + value: 7.513212944963869 + - type: nauc_precision_at_100_std + value: 5.237924977571242 + - type: nauc_precision_at_10_diff1 + value: 6.77138100994466 + - type: nauc_precision_at_10_max + value: 7.376874826886665 + - type: nauc_precision_at_10_std + value: 0.6383203572368118 + - type: nauc_precision_at_1_diff1 + value: 18.54493340747909 + - type: nauc_precision_at_1_max + value: 8.77355237570233 + - type: nauc_precision_at_1_std + value: 5.594302960623576 + - type: nauc_precision_at_20_diff1 + value: 7.542910400303255 + - type: nauc_precision_at_20_max + value: 8.702611395923912 + - type: nauc_precision_at_20_std + value: 3.4101571933438444 + - type: nauc_precision_at_3_diff1 + value: 7.426612756605223 + - type: nauc_precision_at_3_max + value: 4.394162144689597 + - type: nauc_precision_at_3_std + value: 5.339197834739952 + - type: nauc_precision_at_5_diff1 + value: 4.929106492214219 + - type: nauc_precision_at_5_max + value: 4.784302262927969 + - type: nauc_precision_at_5_std + value: -0.030270136453348934 + - type: nauc_recall_at_1000_diff1 + value: 14.352619292360961 + - type: nauc_recall_at_1000_max + value: 9.06600701556724 + - type: nauc_recall_at_1000_std + value: 15.130728840498673 + - type: nauc_recall_at_100_diff1 + value: 20.208981835771443 + - type: nauc_recall_at_100_max + value: 8.008407920519739 + - type: nauc_recall_at_100_std + value: 6.515568441370566 + - type: nauc_recall_at_10_diff1 + value: 10.259953355933666 + - type: nauc_recall_at_10_max + value: 7.361918177568944 + - type: nauc_recall_at_10_std + value: 0.45481413814853 + - type: nauc_recall_at_1_diff1 + value: 20.218709814462795 + - type: nauc_recall_at_1_max + value: 8.036844386597277 + - type: nauc_recall_at_1_std + value: 4.620772690437222 + - type: nauc_recall_at_20_diff1 + value: 10.124366542143687 + - type: nauc_recall_at_20_max + value: 10.273002819827488 + - type: nauc_recall_at_20_std + value: 2.441630654254638 + - type: nauc_recall_at_3_diff1 + value: 9.479158857315646 + - type: nauc_recall_at_3_max + value: 2.895928149511274 + - type: nauc_recall_at_3_std + value: 2.332135182452361 + - type: nauc_recall_at_5_diff1 + value: 7.09949477683149 + - type: nauc_recall_at_5_max + value: 2.4529199558425057 + - type: nauc_recall_at_5_std + value: -2.574411094667869 + - type: ndcg_at_1 + value: 7.2139999999999995 + - type: ndcg_at_10 + value: 11.477 + - type: ndcg_at_100 + value: 15.778 + - type: ndcg_at_1000 + value: 19.335 + - type: ndcg_at_20 + value: 12.884 + - type: ndcg_at_3 + value: 9.097 + - type: ndcg_at_5 + value: 9.959 + - type: precision_at_1 + value: 7.2139999999999995 + - type: precision_at_10 + value: 2.2009999999999996 + - type: precision_at_100 + value: 0.516 + - type: precision_at_1000 + value: 0.096 + - type: precision_at_20 + value: 1.474 + - type: precision_at_3 + value: 4.436 + - type: precision_at_5 + value: 3.209 + - type: recall_at_1 + value: 5.978 + - type: recall_at_10 + value: 17.011000000000003 + - type: recall_at_100 + value: 36.571 + - type: recall_at_1000 + value: 63.019000000000005 + - type: recall_at_20 + value: 22.158 + - type: recall_at_3 + value: 10.395 + - type: recall_at_5 + value: 12.519 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackPhysicsRetrieval (default) + revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4 + split: test + type: mteb/cqadupstack-physics + metrics: + - type: main_score + value: 24.339 + - type: map_at_1 + value: 14.276 + - type: map_at_10 + value: 20.214 + - type: map_at_100 + value: 21.36 + - type: map_at_1000 + value: 21.497 + - type: map_at_20 + value: 20.82 + - type: map_at_3 + value: 18.15 + - type: map_at_5 + value: 19.303 + - type: mrr_at_1 + value: 17.51684311838306 + - type: mrr_at_10 + value: 23.891218662633474 + - type: mrr_at_100 + value: 24.937335828242503 + - type: mrr_at_1000 + value: 25.022891615789916 + - type: mrr_at_20 + value: 24.539336514663326 + - type: mrr_at_3 + value: 21.86397176772539 + - type: mrr_at_5 + value: 22.946743663779277 + - type: nauc_map_at_1000_diff1 + value: 41.03685606915441 + - type: nauc_map_at_1000_max + value: 28.407889486060654 + - type: nauc_map_at_1000_std + value: 0.22648768377295575 + - type: nauc_map_at_100_diff1 + value: 41.062268537451416 + - type: nauc_map_at_100_max + value: 28.349515343592774 + - type: nauc_map_at_100_std + value: 0.17638147423187492 + - type: nauc_map_at_10_diff1 + value: 41.4848494217639 + - type: nauc_map_at_10_max + value: 28.30792618978453 + - type: nauc_map_at_10_std + value: -0.5884871668402413 + - type: nauc_map_at_1_diff1 + value: 51.34490233640957 + - type: nauc_map_at_1_max + value: 32.04388752102519 + - type: nauc_map_at_1_std + value: -2.0244080776431432 + - type: nauc_map_at_20_diff1 + value: 41.011278551472955 + - type: nauc_map_at_20_max + value: 28.17575962730337 + - type: nauc_map_at_20_std + value: -0.22940262771844888 + - type: nauc_map_at_3_diff1 + value: 43.79886641228932 + - type: nauc_map_at_3_max + value: 28.80071801875486 + - type: nauc_map_at_3_std + value: -1.5768826861725582 + - type: nauc_map_at_5_diff1 + value: 42.00099643721408 + - type: nauc_map_at_5_max + value: 28.027991423509725 + - type: nauc_map_at_5_std + value: -1.1865794433594854 + - type: nauc_mrr_at_1000_diff1 + value: 38.36644049601885 + - type: nauc_mrr_at_1000_max + value: 29.20697821315908 + - type: nauc_mrr_at_1000_std + value: 2.98422553488691 + - type: nauc_mrr_at_100_diff1 + value: 38.356819684049654 + - type: nauc_mrr_at_100_max + value: 29.19843618169775 + - type: nauc_mrr_at_100_std + value: 2.9897198518781205 + - type: nauc_mrr_at_10_diff1 + value: 38.5333441146625 + - type: nauc_mrr_at_10_max + value: 29.25137274523934 + - type: nauc_mrr_at_10_std + value: 2.3529920469429233 + - type: nauc_mrr_at_1_diff1 + value: 47.366507524991746 + - type: nauc_mrr_at_1_max + value: 32.895281342019324 + - type: nauc_mrr_at_1_std + value: 1.8030005375326137 + - type: nauc_mrr_at_20_diff1 + value: 38.211103001509194 + - type: nauc_mrr_at_20_max + value: 29.186823583515988 + - type: nauc_mrr_at_20_std + value: 2.812957883224704 + - type: nauc_mrr_at_3_diff1 + value: 40.47065080516183 + - type: nauc_mrr_at_3_max + value: 30.206642160686492 + - type: nauc_mrr_at_3_std + value: 2.1869916923301416 + - type: nauc_mrr_at_5_diff1 + value: 38.94103732935405 + - type: nauc_mrr_at_5_max + value: 29.44193725160551 + - type: nauc_mrr_at_5_std + value: 2.176339260825003 + - type: nauc_ndcg_at_1000_diff1 + value: 36.536198648360866 + - type: nauc_ndcg_at_1000_max + value: 28.33360143898126 + - type: nauc_ndcg_at_1000_std + value: 5.405088279297754 + - type: nauc_ndcg_at_100_diff1 + value: 36.77762883229137 + - type: nauc_ndcg_at_100_max + value: 27.60828107876558 + - type: nauc_ndcg_at_100_std + value: 4.538381997168604 + - type: nauc_ndcg_at_10_diff1 + value: 37.29177848244754 + - type: nauc_ndcg_at_10_max + value: 27.152285083637018 + - type: nauc_ndcg_at_10_std + value: 0.8833331313768573 + - type: nauc_ndcg_at_1_diff1 + value: 47.366507524991746 + - type: nauc_ndcg_at_1_max + value: 32.895281342019324 + - type: nauc_ndcg_at_1_std + value: 1.8030005375326137 + - type: nauc_ndcg_at_20_diff1 + value: 35.898785819812296 + - type: nauc_ndcg_at_20_max + value: 26.741188274092714 + - type: nauc_ndcg_at_20_std + value: 2.128249456208991 + - type: nauc_ndcg_at_3_diff1 + value: 40.604578269692986 + - type: nauc_ndcg_at_3_max + value: 28.921397080144978 + - type: nauc_ndcg_at_3_std + value: 0.2701872778553062 + - type: nauc_ndcg_at_5_diff1 + value: 38.08793641548167 + - type: nauc_ndcg_at_5_max + value: 27.167620873313975 + - type: nauc_ndcg_at_5_std + value: 0.1778091881994553 + - type: nauc_precision_at_1000_diff1 + value: -5.6743074187815985 + - type: nauc_precision_at_1000_max + value: 15.5986515824071 + - type: nauc_precision_at_1000_std + value: 17.134642238195273 + - type: nauc_precision_at_100_diff1 + value: 7.3790050645260035 + - type: nauc_precision_at_100_max + value: 24.489647715333167 + - type: nauc_precision_at_100_std + value: 19.12946172674425 + - type: nauc_precision_at_10_diff1 + value: 20.188506668009886 + - type: nauc_precision_at_10_max + value: 26.777986352425785 + - type: nauc_precision_at_10_std + value: 8.147538901049616 + - type: nauc_precision_at_1_diff1 + value: 47.366507524991746 + - type: nauc_precision_at_1_max + value: 32.895281342019324 + - type: nauc_precision_at_1_std + value: 1.8030005375326137 + - type: nauc_precision_at_20_diff1 + value: 14.0956852447824 + - type: nauc_precision_at_20_max + value: 25.068550907909394 + - type: nauc_precision_at_20_std + value: 12.083382546281381 + - type: nauc_precision_at_3_diff1 + value: 30.61702706951849 + - type: nauc_precision_at_3_max + value: 29.360128712270566 + - type: nauc_precision_at_3_std + value: 5.900367278881995 + - type: nauc_precision_at_5_diff1 + value: 25.03904537180811 + - type: nauc_precision_at_5_max + value: 26.368792424461123 + - type: nauc_precision_at_5_std + value: 5.5938190645224 + - type: nauc_recall_at_1000_diff1 + value: 19.638152372175917 + - type: nauc_recall_at_1000_max + value: 23.177059214668773 + - type: nauc_recall_at_1000_std + value: 32.182698643855346 + - type: nauc_recall_at_100_diff1 + value: 26.549351228167538 + - type: nauc_recall_at_100_max + value: 20.94797530163321 + - type: nauc_recall_at_100_std + value: 15.652166683248641 + - type: nauc_recall_at_10_diff1 + value: 28.31827760081841 + - type: nauc_recall_at_10_max + value: 21.840786780978892 + - type: nauc_recall_at_10_std + value: 1.7870205785383046 + - type: nauc_recall_at_1_diff1 + value: 51.34490233640957 + - type: nauc_recall_at_1_max + value: 32.04388752102519 + - type: nauc_recall_at_1_std + value: -2.0244080776431432 + - type: nauc_recall_at_20_diff1 + value: 23.491682404438986 + - type: nauc_recall_at_20_max + value: 19.86816079086787 + - type: nauc_recall_at_20_std + value: 5.25218609094941 + - type: nauc_recall_at_3_diff1 + value: 35.96141633078342 + - type: nauc_recall_at_3_max + value: 24.814199432686003 + - type: nauc_recall_at_3_std + value: -0.3476350788514478 + - type: nauc_recall_at_5_diff1 + value: 30.5146792354956 + - type: nauc_recall_at_5_max + value: 22.120397499890533 + - type: nauc_recall_at_5_std + value: 0.2403134576449495 + - type: ndcg_at_1 + value: 17.517 + - type: ndcg_at_10 + value: 24.339 + - type: ndcg_at_100 + value: 30.032999999999998 + - type: ndcg_at_1000 + value: 33.23 + - type: ndcg_at_20 + value: 26.455000000000002 + - type: ndcg_at_3 + value: 20.65 + - type: ndcg_at_5 + value: 22.356 + - type: precision_at_1 + value: 17.517 + - type: precision_at_10 + value: 4.668 + - type: precision_at_100 + value: 0.8999999999999999 + - type: precision_at_1000 + value: 0.13799999999999998 + - type: precision_at_20 + value: 2.94 + - type: precision_at_3 + value: 9.913 + - type: precision_at_5 + value: 7.276000000000001 + - type: recall_at_1 + value: 14.276 + - type: recall_at_10 + value: 33.101 + - type: recall_at_100 + value: 58.28 + - type: recall_at_1000 + value: 80.322 + - type: recall_at_20 + value: 40.811 + - type: recall_at_3 + value: 22.708000000000002 + - type: recall_at_5 + value: 27.126 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackProgrammersRetrieval (default) + revision: 6184bc1440d2dbc7612be22b50686b8826d22b32 + split: test + type: mteb/cqadupstack-programmers + metrics: + - type: main_score + value: 18.572 + - type: map_at_1 + value: 10.113999999999999 + - type: map_at_10 + value: 14.966 + - type: map_at_100 + value: 16.142 + - type: map_at_1000 + value: 16.274 + - type: map_at_20 + value: 15.588 + - type: map_at_3 + value: 13.306000000000001 + - type: map_at_5 + value: 14.115 + - type: mrr_at_1 + value: 12.32876712328767 + - type: mrr_at_10 + value: 18.100311661955487 + - type: mrr_at_100 + value: 19.104690734771953 + - type: mrr_at_1000 + value: 19.191702866697273 + - type: mrr_at_20 + value: 18.642865880622466 + - type: mrr_at_3 + value: 16.229071537290704 + - type: mrr_at_5 + value: 17.085235920852355 + - type: nauc_map_at_1000_diff1 + value: 31.5380839663444 + - type: nauc_map_at_1000_max + value: 17.9681008389522 + - type: nauc_map_at_1000_std + value: 2.2164295402511014 + - type: nauc_map_at_100_diff1 + value: 31.549568702174767 + - type: nauc_map_at_100_max + value: 17.875779141578462 + - type: nauc_map_at_100_std + value: 2.0933869829236347 + - type: nauc_map_at_10_diff1 + value: 32.19172260750653 + - type: nauc_map_at_10_max + value: 17.306789949301294 + - type: nauc_map_at_10_std + value: 0.700237138708676 + - type: nauc_map_at_1_diff1 + value: 41.99338272552499 + - type: nauc_map_at_1_max + value: 19.41453494529689 + - type: nauc_map_at_1_std + value: -2.9915157281415703 + - type: nauc_map_at_20_diff1 + value: 31.73238374507595 + - type: nauc_map_at_20_max + value: 17.311580430203353 + - type: nauc_map_at_20_std + value: 1.5238571528713525 + - type: nauc_map_at_3_diff1 + value: 32.949765145983314 + - type: nauc_map_at_3_max + value: 16.284348173129974 + - type: nauc_map_at_3_std + value: -1.3141721735461476 + - type: nauc_map_at_5_diff1 + value: 31.963187574971176 + - type: nauc_map_at_5_max + value: 17.100787287868354 + - type: nauc_map_at_5_std + value: 0.0015731783817020976 + - type: nauc_mrr_at_1000_diff1 + value: 28.148462053021166 + - type: nauc_mrr_at_1000_max + value: 19.28312681351677 + - type: nauc_mrr_at_1000_std + value: 3.8584952061086435 + - type: nauc_mrr_at_100_diff1 + value: 28.094200856177288 + - type: nauc_mrr_at_100_max + value: 19.24299141244573 + - type: nauc_mrr_at_100_std + value: 3.8382724028400093 + - type: nauc_mrr_at_10_diff1 + value: 28.51829789033651 + - type: nauc_mrr_at_10_max + value: 19.447637292790958 + - type: nauc_mrr_at_10_std + value: 3.19294049353816 + - type: nauc_mrr_at_1_diff1 + value: 39.31634352935456 + - type: nauc_mrr_at_1_max + value: 21.789914492317912 + - type: nauc_mrr_at_1_std + value: -0.22812038812251473 + - type: nauc_mrr_at_20_diff1 + value: 28.17131515128093 + - type: nauc_mrr_at_20_max + value: 18.954274621475932 + - type: nauc_mrr_at_20_std + value: 3.5798521757707897 + - type: nauc_mrr_at_3_diff1 + value: 29.622155411044332 + - type: nauc_mrr_at_3_max + value: 19.238666216526838 + - type: nauc_mrr_at_3_std + value: 1.8906405763669278 + - type: nauc_mrr_at_5_diff1 + value: 28.774014587372527 + - type: nauc_mrr_at_5_max + value: 19.414140006144585 + - type: nauc_mrr_at_5_std + value: 2.840339018057259 + - type: nauc_ndcg_at_1000_diff1 + value: 26.895080768863867 + - type: nauc_ndcg_at_1000_max + value: 19.684613860376647 + - type: nauc_ndcg_at_1000_std + value: 9.793487660604164 + - type: nauc_ndcg_at_100_diff1 + value: 26.78733090807618 + - type: nauc_ndcg_at_100_max + value: 18.91812508266874 + - type: nauc_ndcg_at_100_std + value: 8.32820999415013 + - type: nauc_ndcg_at_10_diff1 + value: 28.51747551047639 + - type: nauc_ndcg_at_10_max + value: 17.451523037784746 + - type: nauc_ndcg_at_10_std + value: 3.4468383889800123 + - type: nauc_ndcg_at_1_diff1 + value: 39.31634352935456 + - type: nauc_ndcg_at_1_max + value: 21.789914492317912 + - type: nauc_ndcg_at_1_std + value: -0.22812038812251473 + - type: nauc_ndcg_at_20_diff1 + value: 27.280597584151316 + - type: nauc_ndcg_at_20_max + value: 16.67121677588379 + - type: nauc_ndcg_at_20_std + value: 5.313461111433228 + - type: nauc_ndcg_at_3_diff1 + value: 29.268509392372348 + - type: nauc_ndcg_at_3_max + value: 16.68884126553042 + - type: nauc_ndcg_at_3_std + value: 0.3627021483553513 + - type: nauc_ndcg_at_5_diff1 + value: 28.083836200902834 + - type: nauc_ndcg_at_5_max + value: 17.611782342073173 + - type: nauc_ndcg_at_5_std + value: 2.2512763000110536 + - type: nauc_precision_at_1000_diff1 + value: -7.4970098056305945 + - type: nauc_precision_at_1000_max + value: 12.162352895422861 + - type: nauc_precision_at_1000_std + value: 16.211487627542326 + - type: nauc_precision_at_100_diff1 + value: 4.007717730230379 + - type: nauc_precision_at_100_max + value: 20.704245855550486 + - type: nauc_precision_at_100_std + value: 21.96869724772107 + - type: nauc_precision_at_10_diff1 + value: 15.831711280362699 + - type: nauc_precision_at_10_max + value: 20.001368457446638 + - type: nauc_precision_at_10_std + value: 10.737927827381192 + - type: nauc_precision_at_1_diff1 + value: 39.31634352935456 + - type: nauc_precision_at_1_max + value: 21.789914492317912 + - type: nauc_precision_at_1_std + value: -0.22812038812251473 + - type: nauc_precision_at_20_diff1 + value: 11.205148892259576 + - type: nauc_precision_at_20_max + value: 18.327783013272807 + - type: nauc_precision_at_20_std + value: 15.41328854851903 + - type: nauc_precision_at_3_diff1 + value: 20.151568649441437 + - type: nauc_precision_at_3_max + value: 18.1027955515208 + - type: nauc_precision_at_3_std + value: 3.8969218089403976 + - type: nauc_precision_at_5_diff1 + value: 15.735602797404976 + - type: nauc_precision_at_5_max + value: 20.417869995880807 + - type: nauc_precision_at_5_std + value: 7.859648888047052 + - type: nauc_recall_at_1000_diff1 + value: 17.091378487075364 + - type: nauc_recall_at_1000_max + value: 22.040731541942677 + - type: nauc_recall_at_1000_std + value: 39.444689096986544 + - type: nauc_recall_at_100_diff1 + value: 18.17709025244327 + - type: nauc_recall_at_100_max + value: 18.312516335450827 + - type: nauc_recall_at_100_std + value: 21.80943236545304 + - type: nauc_recall_at_10_diff1 + value: 23.21820368922194 + - type: nauc_recall_at_10_max + value: 15.040119533520468 + - type: nauc_recall_at_10_std + value: 7.412805177818517 + - type: nauc_recall_at_1_diff1 + value: 41.99338272552499 + - type: nauc_recall_at_1_max + value: 19.41453494529689 + - type: nauc_recall_at_1_std + value: -2.9915157281415703 + - type: nauc_recall_at_20_diff1 + value: 19.927391383187544 + - type: nauc_recall_at_20_max + value: 12.10764205758219 + - type: nauc_recall_at_20_std + value: 11.456305933505394 + - type: nauc_recall_at_3_diff1 + value: 24.57783329903588 + - type: nauc_recall_at_3_max + value: 13.251921475093864 + - type: nauc_recall_at_3_std + value: 0.4602962145435756 + - type: nauc_recall_at_5_diff1 + value: 22.7513858055345 + - type: nauc_recall_at_5_max + value: 15.317660220264273 + - type: nauc_recall_at_5_std + value: 4.844790346315235 + - type: ndcg_at_1 + value: 12.328999999999999 + - type: ndcg_at_10 + value: 18.572 + - type: ndcg_at_100 + value: 24.189 + - type: ndcg_at_1000 + value: 27.564 + - type: ndcg_at_20 + value: 20.580000000000002 + - type: ndcg_at_3 + value: 15.346000000000002 + - type: ndcg_at_5 + value: 16.567999999999998 + - type: precision_at_1 + value: 12.328999999999999 + - type: precision_at_10 + value: 3.6990000000000003 + - type: precision_at_100 + value: 0.788 + - type: precision_at_1000 + value: 0.125 + - type: precision_at_20 + value: 2.443 + - type: precision_at_3 + value: 7.61 + - type: precision_at_5 + value: 5.548 + - type: recall_at_1 + value: 10.113999999999999 + - type: recall_at_10 + value: 26.299 + - type: recall_at_100 + value: 50.963 + - type: recall_at_1000 + value: 75.40400000000001 + - type: recall_at_20 + value: 33.398 + - type: recall_at_3 + value: 17.283 + - type: recall_at_5 + value: 20.448 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackRetrieval (default) + revision: CQADupstackRetrieval_is_a_combined_dataset + split: test + type: CQADupstackRetrieval_is_a_combined_dataset + metrics: + - type: main_score + value: 20.06633333333333 + - type: ndcg_at_10 + value: 20.06633333333333 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackStatsRetrieval (default) + revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a + split: test + type: mteb/cqadupstack-stats + metrics: + - type: main_score + value: 14.771999999999998 + - type: map_at_1 + value: 7.9670000000000005 + - type: map_at_10 + value: 11.916 + - type: map_at_100 + value: 12.672 + - type: map_at_1000 + value: 12.756 + - type: map_at_20 + value: 12.313 + - type: map_at_3 + value: 10.445 + - type: map_at_5 + value: 11.274000000000001 + - type: mrr_at_1 + value: 9.662576687116564 + - type: mrr_at_10 + value: 13.881889667932617 + - type: mrr_at_100 + value: 14.622003040491299 + - type: mrr_at_1000 + value: 14.700421998539593 + - type: mrr_at_20 + value: 14.26597490297063 + - type: mrr_at_3 + value: 12.295501022494886 + - type: mrr_at_5 + value: 13.185071574642123 + - type: nauc_map_at_1000_diff1 + value: 27.464956379343636 + - type: nauc_map_at_1000_max + value: 12.688799707158202 + - type: nauc_map_at_1000_std + value: 6.181513833234629 + - type: nauc_map_at_100_diff1 + value: 27.480291541621977 + - type: nauc_map_at_100_max + value: 12.669898754417847 + - type: nauc_map_at_100_std + value: 6.161789467868528 + - type: nauc_map_at_10_diff1 + value: 27.84183710063236 + - type: nauc_map_at_10_max + value: 13.103624223429215 + - type: nauc_map_at_10_std + value: 5.826931197085812 + - type: nauc_map_at_1_diff1 + value: 35.26968831241412 + - type: nauc_map_at_1_max + value: 13.821463026692879 + - type: nauc_map_at_1_std + value: 2.6240801068368147 + - type: nauc_map_at_20_diff1 + value: 27.694640221569394 + - type: nauc_map_at_20_max + value: 12.566821566355408 + - type: nauc_map_at_20_std + value: 5.841018047634929 + - type: nauc_map_at_3_diff1 + value: 29.554875884424263 + - type: nauc_map_at_3_max + value: 14.305644406759505 + - type: nauc_map_at_3_std + value: 5.601041654228171 + - type: nauc_map_at_5_diff1 + value: 29.240061587938037 + - type: nauc_map_at_5_max + value: 14.00128435019755 + - type: nauc_map_at_5_std + value: 5.82150408034715 + - type: nauc_mrr_at_1000_diff1 + value: 27.17619540994222 + - type: nauc_mrr_at_1000_max + value: 14.59029391462989 + - type: nauc_mrr_at_1000_std + value: 7.36596867864775 + - type: nauc_mrr_at_100_diff1 + value: 27.168129068852892 + - type: nauc_mrr_at_100_max + value: 14.547185893357803 + - type: nauc_mrr_at_100_std + value: 7.356270166890706 + - type: nauc_mrr_at_10_diff1 + value: 27.5014826968088 + - type: nauc_mrr_at_10_max + value: 14.986374849985179 + - type: nauc_mrr_at_10_std + value: 7.098527350761 + - type: nauc_mrr_at_1_diff1 + value: 34.74758694269618 + - type: nauc_mrr_at_1_max + value: 15.94870709819082 + - type: nauc_mrr_at_1_std + value: 3.484756136581587 + - type: nauc_mrr_at_20_diff1 + value: 27.30678702098046 + - type: nauc_mrr_at_20_max + value: 14.64068189163014 + - type: nauc_mrr_at_20_std + value: 7.222167908721225 + - type: nauc_mrr_at_3_diff1 + value: 29.05886648272905 + - type: nauc_mrr_at_3_max + value: 16.437400286007918 + - type: nauc_mrr_at_3_std + value: 7.033090959882006 + - type: nauc_mrr_at_5_diff1 + value: 28.75634825543908 + - type: nauc_mrr_at_5_max + value: 16.116487363272213 + - type: nauc_mrr_at_5_std + value: 7.357847583847684 + - type: nauc_ndcg_at_1000_diff1 + value: 23.407580241714584 + - type: nauc_ndcg_at_1000_max + value: 11.773215821924362 + - type: nauc_ndcg_at_1000_std + value: 9.058784561916143 + - type: nauc_ndcg_at_100_diff1 + value: 23.979537088722516 + - type: nauc_ndcg_at_100_max + value: 11.295731197485436 + - type: nauc_ndcg_at_100_std + value: 8.714374001997927 + - type: nauc_ndcg_at_10_diff1 + value: 24.759023148739058 + - type: nauc_ndcg_at_10_max + value: 12.341084446128495 + - type: nauc_ndcg_at_10_std + value: 6.848529153224625 + - type: nauc_ndcg_at_1_diff1 + value: 34.74758694269618 + - type: nauc_ndcg_at_1_max + value: 15.94870709819082 + - type: nauc_ndcg_at_1_std + value: 3.484756136581587 + - type: nauc_ndcg_at_20_diff1 + value: 24.647169467462906 + - type: nauc_ndcg_at_20_max + value: 10.81220813658661 + - type: nauc_ndcg_at_20_std + value: 7.1325252724340595 + - type: nauc_ndcg_at_3_diff1 + value: 28.251959160053886 + - type: nauc_ndcg_at_3_max + value: 14.534129576164926 + - type: nauc_ndcg_at_3_std + value: 6.919835773799475 + - type: nauc_ndcg_at_5_diff1 + value: 27.767475859457825 + - type: nauc_ndcg_at_5_max + value: 14.308510586980146 + - type: nauc_ndcg_at_5_std + value: 6.997517881896331 + - type: nauc_precision_at_1000_diff1 + value: 8.890773326647109 + - type: nauc_precision_at_1000_max + value: 12.287863157289907 + - type: nauc_precision_at_1000_std + value: 17.835978900903893 + - type: nauc_precision_at_100_diff1 + value: 16.784612912355854 + - type: nauc_precision_at_100_max + value: 13.354526754596558 + - type: nauc_precision_at_100_std + value: 18.18769011052455 + - type: nauc_precision_at_10_diff1 + value: 18.949749011392175 + - type: nauc_precision_at_10_max + value: 12.633074463041934 + - type: nauc_precision_at_10_std + value: 11.69597288682816 + - type: nauc_precision_at_1_diff1 + value: 34.74758694269618 + - type: nauc_precision_at_1_max + value: 15.94870709819082 + - type: nauc_precision_at_1_std + value: 3.484756136581587 + - type: nauc_precision_at_20_diff1 + value: 19.871714092445643 + - type: nauc_precision_at_20_max + value: 9.369266922594758 + - type: nauc_precision_at_20_std + value: 12.215660461035935 + - type: nauc_precision_at_3_diff1 + value: 24.779708502791607 + - type: nauc_precision_at_3_max + value: 17.242869556602585 + - type: nauc_precision_at_3_std + value: 10.535864690942665 + - type: nauc_precision_at_5_diff1 + value: 23.983801571912284 + - type: nauc_precision_at_5_max + value: 17.366696818116218 + - type: nauc_precision_at_5_std + value: 11.873524889936338 + - type: nauc_recall_at_1000_diff1 + value: 12.066728664027488 + - type: nauc_recall_at_1000_max + value: 7.337077809378563 + - type: nauc_recall_at_1000_std + value: 14.409509010648286 + - type: nauc_recall_at_100_diff1 + value: 16.485850144126058 + - type: nauc_recall_at_100_max + value: 5.940924198197424 + - type: nauc_recall_at_100_std + value: 12.53633827092492 + - type: nauc_recall_at_10_diff1 + value: 17.108201444307365 + - type: nauc_recall_at_10_max + value: 8.094878456313593 + - type: nauc_recall_at_10_std + value: 6.922028971621734 + - type: nauc_recall_at_1_diff1 + value: 35.26968831241412 + - type: nauc_recall_at_1_max + value: 13.821463026692879 + - type: nauc_recall_at_1_std + value: 2.6240801068368147 + - type: nauc_recall_at_20_diff1 + value: 17.770237163799866 + - type: nauc_recall_at_20_max + value: 4.153335758130064 + - type: nauc_recall_at_20_std + value: 7.810562339838044 + - type: nauc_recall_at_3_diff1 + value: 24.528686523846783 + - type: nauc_recall_at_3_max + value: 13.260197733449864 + - type: nauc_recall_at_3_std + value: 7.236438838203539 + - type: nauc_recall_at_5_diff1 + value: 24.526787737523158 + - type: nauc_recall_at_5_max + value: 12.931129112159464 + - type: nauc_recall_at_5_std + value: 7.69306537628985 + - type: ndcg_at_1 + value: 9.663 + - type: ndcg_at_10 + value: 14.771999999999998 + - type: ndcg_at_100 + value: 18.804000000000002 + - type: ndcg_at_1000 + value: 21.398 + - type: ndcg_at_20 + value: 16.158 + - type: ndcg_at_3 + value: 11.826 + - type: ndcg_at_5 + value: 13.22 + - type: precision_at_1 + value: 9.663 + - type: precision_at_10 + value: 2.638 + - type: precision_at_100 + value: 0.506 + - type: precision_at_1000 + value: 0.079 + - type: precision_at_20 + value: 1.641 + - type: precision_at_3 + value: 5.573 + - type: precision_at_5 + value: 4.141 + - type: recall_at_1 + value: 7.9670000000000005 + - type: recall_at_10 + value: 21.95 + - type: recall_at_100 + value: 40.909 + - type: recall_at_1000 + value: 60.781 + - type: recall_at_20 + value: 27.235 + - type: recall_at_3 + value: 13.639000000000001 + - type: recall_at_5 + value: 17.197000000000003 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackTexRetrieval (default) + revision: 46989137a86843e03a6195de44b09deda022eec7 + split: test + type: mteb/cqadupstack-tex + metrics: + - type: main_score + value: 11.89 + - type: map_at_1 + value: 6.544 + - type: map_at_10 + value: 9.652 + - type: map_at_100 + value: 10.302999999999999 + - type: map_at_1000 + value: 10.419 + - type: map_at_20 + value: 10.012 + - type: map_at_3 + value: 8.602 + - type: map_at_5 + value: 9.148 + - type: mrr_at_1 + value: 7.91465932553338 + - type: mrr_at_10 + value: 11.574451872972155 + - type: mrr_at_100 + value: 12.26293448933862 + - type: mrr_at_1000 + value: 12.35992343686757 + - type: mrr_at_20 + value: 11.962637194186767 + - type: mrr_at_3 + value: 10.329203945859144 + - type: mrr_at_5 + value: 10.955494379444817 + - type: nauc_map_at_1000_diff1 + value: 29.599569655376 + - type: nauc_map_at_1000_max + value: 14.260050539335332 + - type: nauc_map_at_1000_std + value: -3.6638649700127686 + - type: nauc_map_at_100_diff1 + value: 29.626869833183132 + - type: nauc_map_at_100_max + value: 14.188784208136356 + - type: nauc_map_at_100_std + value: -3.7889138601146852 + - type: nauc_map_at_10_diff1 + value: 30.240601138831547 + - type: nauc_map_at_10_max + value: 13.916837722756387 + - type: nauc_map_at_10_std + value: -4.316378076446822 + - type: nauc_map_at_1_diff1 + value: 38.49732191352995 + - type: nauc_map_at_1_max + value: 13.959321672081417 + - type: nauc_map_at_1_std + value: -5.405022381926919 + - type: nauc_map_at_20_diff1 + value: 29.869505981506343 + - type: nauc_map_at_20_max + value: 14.009382089208437 + - type: nauc_map_at_20_std + value: -4.193680132404101 + - type: nauc_map_at_3_diff1 + value: 32.880929152413465 + - type: nauc_map_at_3_max + value: 14.7218073741984 + - type: nauc_map_at_3_std + value: -4.0556191052631965 + - type: nauc_map_at_5_diff1 + value: 31.269904437209846 + - type: nauc_map_at_5_max + value: 13.937943418680376 + - type: nauc_map_at_5_std + value: -4.354961218325039 + - type: nauc_mrr_at_1000_diff1 + value: 28.28534237979384 + - type: nauc_mrr_at_1000_max + value: 15.617476454684626 + - type: nauc_mrr_at_1000_std + value: -3.121517210162604 + - type: nauc_mrr_at_100_diff1 + value: 28.28445731466983 + - type: nauc_mrr_at_100_max + value: 15.598018147875806 + - type: nauc_mrr_at_100_std + value: -3.1630011896008376 + - type: nauc_mrr_at_10_diff1 + value: 28.720037201906628 + - type: nauc_mrr_at_10_max + value: 15.446326925464204 + - type: nauc_mrr_at_10_std + value: -3.628030605618028 + - type: nauc_mrr_at_1_diff1 + value: 37.213053214984974 + - type: nauc_mrr_at_1_max + value: 16.871596517680164 + - type: nauc_mrr_at_1_std + value: -5.866424771722043 + - type: nauc_mrr_at_20_diff1 + value: 28.4289942313558 + - type: nauc_mrr_at_20_max + value: 15.526335575515372 + - type: nauc_mrr_at_20_std + value: -3.4399906991625007 + - type: nauc_mrr_at_3_diff1 + value: 31.457735295395793 + - type: nauc_mrr_at_3_max + value: 16.14296947486417 + - type: nauc_mrr_at_3_std + value: -4.090661296421823 + - type: nauc_mrr_at_5_diff1 + value: 29.83938535974195 + - type: nauc_mrr_at_5_max + value: 15.41160305310037 + - type: nauc_mrr_at_5_std + value: -3.8949872883203978 + - type: nauc_ndcg_at_1000_diff1 + value: 24.03973945083958 + - type: nauc_ndcg_at_1000_max + value: 15.990800342336264 + - type: nauc_ndcg_at_1000_std + value: 1.15168031596194 + - type: nauc_ndcg_at_100_diff1 + value: 24.46192215199325 + - type: nauc_ndcg_at_100_max + value: 14.6392676264499 + - type: nauc_ndcg_at_100_std + value: -1.076913377988602 + - type: nauc_ndcg_at_10_diff1 + value: 26.544537940077305 + - type: nauc_ndcg_at_10_max + value: 13.741302138379494 + - type: nauc_ndcg_at_10_std + value: -3.792408046032157 + - type: nauc_ndcg_at_1_diff1 + value: 37.213053214984974 + - type: nauc_ndcg_at_1_max + value: 16.871596517680164 + - type: nauc_ndcg_at_1_std + value: -5.866424771722043 + - type: nauc_ndcg_at_20_diff1 + value: 25.5102221223638 + - type: nauc_ndcg_at_20_max + value: 13.990598985829614 + - type: nauc_ndcg_at_20_std + value: -3.368720146146748 + - type: nauc_ndcg_at_3_diff1 + value: 30.989600349929447 + - type: nauc_ndcg_at_3_max + value: 15.226808763832569 + - type: nauc_ndcg_at_3_std + value: -3.601126257016185 + - type: nauc_ndcg_at_5_diff1 + value: 28.54393105410386 + - type: nauc_ndcg_at_5_max + value: 13.792435440713641 + - type: nauc_ndcg_at_5_std + value: -3.942970556452405 + - type: nauc_precision_at_1000_diff1 + value: 8.397248790713299 + - type: nauc_precision_at_1000_max + value: 22.865216579494238 + - type: nauc_precision_at_1000_std + value: 18.964105556706816 + - type: nauc_precision_at_100_diff1 + value: 13.795302908729973 + - type: nauc_precision_at_100_max + value: 19.894620163243133 + - type: nauc_precision_at_100_std + value: 9.265664986819248 + - type: nauc_precision_at_10_diff1 + value: 18.894866608417136 + - type: nauc_precision_at_10_max + value: 16.174625648208707 + - type: nauc_precision_at_10_std + value: -1.748849701509255 + - type: nauc_precision_at_1_diff1 + value: 37.213053214984974 + - type: nauc_precision_at_1_max + value: 16.871596517680164 + - type: nauc_precision_at_1_std + value: -5.866424771722043 + - type: nauc_precision_at_20_diff1 + value: 16.725759138449074 + - type: nauc_precision_at_20_max + value: 18.07640605163822 + - type: nauc_precision_at_20_std + value: 0.8677700106490629 + - type: nauc_precision_at_3_diff1 + value: 27.631950409324226 + - type: nauc_precision_at_3_max + value: 17.730161146279748 + - type: nauc_precision_at_3_std + value: -2.8349313445582793 + - type: nauc_precision_at_5_diff1 + value: 23.092797668276425 + - type: nauc_precision_at_5_max + value: 15.754595059066414 + - type: nauc_precision_at_5_std + value: -2.6981835587673078 + - type: nauc_recall_at_1000_diff1 + value: 11.611126009611255 + - type: nauc_recall_at_1000_max + value: 18.364258718711064 + - type: nauc_recall_at_1000_std + value: 14.590227954099502 + - type: nauc_recall_at_100_diff1 + value: 14.602946067925245 + - type: nauc_recall_at_100_max + value: 13.19041135076776 + - type: nauc_recall_at_100_std + value: 4.279012994387592 + - type: nauc_recall_at_10_diff1 + value: 19.8109475851167 + - type: nauc_recall_at_10_max + value: 11.628627287295066 + - type: nauc_recall_at_10_std + value: -3.3022579440673745 + - type: nauc_recall_at_1_diff1 + value: 38.49732191352995 + - type: nauc_recall_at_1_max + value: 13.959321672081417 + - type: nauc_recall_at_1_std + value: -5.405022381926919 + - type: nauc_recall_at_20_diff1 + value: 17.09822858202577 + - type: nauc_recall_at_20_max + value: 11.883568391126285 + - type: nauc_recall_at_20_std + value: -2.5468000044892034 + - type: nauc_recall_at_3_diff1 + value: 27.497580747993382 + - type: nauc_recall_at_3_max + value: 13.739587046904262 + - type: nauc_recall_at_3_std + value: -2.958549474787635 + - type: nauc_recall_at_5_diff1 + value: 23.200480313988866 + - type: nauc_recall_at_5_max + value: 11.409453989148197 + - type: nauc_recall_at_5_std + value: -3.802220387794951 + - type: ndcg_at_1 + value: 7.915 + - type: ndcg_at_10 + value: 11.89 + - type: ndcg_at_100 + value: 15.409 + - type: ndcg_at_1000 + value: 18.848000000000003 + - type: ndcg_at_20 + value: 13.181999999999999 + - type: ndcg_at_3 + value: 9.783 + - type: ndcg_at_5 + value: 10.665 + - type: precision_at_1 + value: 7.915 + - type: precision_at_10 + value: 2.254 + - type: precision_at_100 + value: 0.47800000000000004 + - type: precision_at_1000 + value: 0.093 + - type: precision_at_20 + value: 1.481 + - type: precision_at_3 + value: 4.691 + - type: precision_at_5 + value: 3.4479999999999995 + - type: recall_at_1 + value: 6.544 + - type: recall_at_10 + value: 16.874 + - type: recall_at_100 + value: 33.027 + - type: recall_at_1000 + value: 58.69 + - type: recall_at_20 + value: 21.676000000000002 + - type: recall_at_3 + value: 11.006 + - type: recall_at_5 + value: 13.242999999999999 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackUnixRetrieval (default) + revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53 + split: test + type: mteb/cqadupstack-unix + metrics: + - type: main_score + value: 17.681 + - type: map_at_1 + value: 10.693 + - type: map_at_10 + value: 14.815000000000001 + - type: map_at_100 + value: 15.562999999999999 + - type: map_at_1000 + value: 15.690999999999999 + - type: map_at_20 + value: 15.198 + - type: map_at_3 + value: 13.492 + - type: map_at_5 + value: 14.251 + - type: mrr_at_1 + value: 12.593283582089551 + - type: mrr_at_10 + value: 17.294220859985778 + - type: mrr_at_100 + value: 18.073301370312517 + - type: mrr_at_1000 + value: 18.178539747275277 + - type: mrr_at_20 + value: 17.726279044649704 + - type: mrr_at_3 + value: 15.889303482587072 + - type: mrr_at_5 + value: 16.654228855721396 + - type: nauc_map_at_1000_diff1 + value: 37.06808689890735 + - type: nauc_map_at_1000_max + value: 25.830253602678106 + - type: nauc_map_at_1000_std + value: -8.200403962720742 + - type: nauc_map_at_100_diff1 + value: 37.10963353228084 + - type: nauc_map_at_100_max + value: 25.79124583385762 + - type: nauc_map_at_100_std + value: -8.305867858237091 + - type: nauc_map_at_10_diff1 + value: 37.55605713097813 + - type: nauc_map_at_10_max + value: 25.97236325257699 + - type: nauc_map_at_10_std + value: -8.787411817254986 + - type: nauc_map_at_1_diff1 + value: 46.880104967892095 + - type: nauc_map_at_1_max + value: 30.47076747044864 + - type: nauc_map_at_1_std + value: -9.529043223192849 + - type: nauc_map_at_20_diff1 + value: 37.40781191515129 + - type: nauc_map_at_20_max + value: 25.87534241343804 + - type: nauc_map_at_20_std + value: -8.727450039123232 + - type: nauc_map_at_3_diff1 + value: 39.39543088322907 + - type: nauc_map_at_3_max + value: 27.250096581190846 + - type: nauc_map_at_3_std + value: -9.1583459940284 + - type: nauc_map_at_5_diff1 + value: 38.37362606408328 + - type: nauc_map_at_5_max + value: 26.49488865613181 + - type: nauc_map_at_5_std + value: -9.204761225287667 + - type: nauc_mrr_at_1000_diff1 + value: 37.35072657075773 + - type: nauc_mrr_at_1000_max + value: 27.103353599284386 + - type: nauc_mrr_at_1000_std + value: -6.772476592783599 + - type: nauc_mrr_at_100_diff1 + value: 37.37461007495732 + - type: nauc_mrr_at_100_max + value: 27.10188269424093 + - type: nauc_mrr_at_100_std + value: -6.838551505670108 + - type: nauc_mrr_at_10_diff1 + value: 37.7609241149103 + - type: nauc_mrr_at_10_max + value: 27.273600798878643 + - type: nauc_mrr_at_10_std + value: -7.288619333100662 + - type: nauc_mrr_at_1_diff1 + value: 48.30751770509831 + - type: nauc_mrr_at_1_max + value: 32.52817497563151 + - type: nauc_mrr_at_1_std + value: -7.5523189586103605 + - type: nauc_mrr_at_20_diff1 + value: 37.5902053059747 + - type: nauc_mrr_at_20_max + value: 27.180853168677206 + - type: nauc_mrr_at_20_std + value: -7.199015767371499 + - type: nauc_mrr_at_3_diff1 + value: 39.423663262845075 + - type: nauc_mrr_at_3_max + value: 28.39283909032157 + - type: nauc_mrr_at_3_std + value: -7.490109392356441 + - type: nauc_mrr_at_5_diff1 + value: 38.584599728860745 + - type: nauc_mrr_at_5_max + value: 27.588748981783258 + - type: nauc_mrr_at_5_std + value: -7.693201065125161 + - type: nauc_ndcg_at_1000_diff1 + value: 31.12351398623868 + - type: nauc_ndcg_at_1000_max + value: 24.13093684291217 + - type: nauc_ndcg_at_1000_std + value: -2.9280612065592444 + - type: nauc_ndcg_at_100_diff1 + value: 32.0198970741007 + - type: nauc_ndcg_at_100_max + value: 23.428381936309968 + - type: nauc_ndcg_at_100_std + value: -5.502846823665347 + - type: nauc_ndcg_at_10_diff1 + value: 34.23942582280388 + - type: nauc_ndcg_at_10_max + value: 24.425351771280987 + - type: nauc_ndcg_at_10_std + value: -7.789211233356857 + - type: nauc_ndcg_at_1_diff1 + value: 48.30751770509831 + - type: nauc_ndcg_at_1_max + value: 32.52817497563151 + - type: nauc_ndcg_at_1_std + value: -7.5523189586103605 + - type: nauc_ndcg_at_20_diff1 + value: 33.89493526795971 + - type: nauc_ndcg_at_20_max + value: 24.267224154957585 + - type: nauc_ndcg_at_20_std + value: -7.516908959235173 + - type: nauc_ndcg_at_3_diff1 + value: 37.20970459712713 + - type: nauc_ndcg_at_3_max + value: 26.89486636826351 + - type: nauc_ndcg_at_3_std + value: -8.276335845643223 + - type: nauc_ndcg_at_5_diff1 + value: 35.795208970782824 + - type: nauc_ndcg_at_5_max + value: 25.45320266356439 + - type: nauc_ndcg_at_5_std + value: -8.649222494447436 + - type: nauc_precision_at_1000_diff1 + value: 0.2021396432488509 + - type: nauc_precision_at_1000_max + value: 15.363228448365076 + - type: nauc_precision_at_1000_std + value: 16.823478933730968 + - type: nauc_precision_at_100_diff1 + value: 13.026435394105418 + - type: nauc_precision_at_100_max + value: 17.52255452715734 + - type: nauc_precision_at_100_std + value: 4.822414814608969 + - type: nauc_precision_at_10_diff1 + value: 24.130000342389195 + - type: nauc_precision_at_10_max + value: 21.21881750694888 + - type: nauc_precision_at_10_std + value: -3.45102769344446 + - type: nauc_precision_at_1_diff1 + value: 48.30751770509831 + - type: nauc_precision_at_1_max + value: 32.52817497563151 + - type: nauc_precision_at_1_std + value: -7.5523189586103605 + - type: nauc_precision_at_20_diff1 + value: 22.452615307016845 + - type: nauc_precision_at_20_max + value: 20.72785119139948 + - type: nauc_precision_at_20_std + value: -2.7892640432250544 + - type: nauc_precision_at_3_diff1 + value: 31.573746476321247 + - type: nauc_precision_at_3_max + value: 25.43799077463197 + - type: nauc_precision_at_3_std + value: -5.584412040269154 + - type: nauc_precision_at_5_diff1 + value: 28.50068432061233 + - type: nauc_precision_at_5_max + value: 23.141781646564937 + - type: nauc_precision_at_5_std + value: -6.256362269388643 + - type: nauc_recall_at_1000_diff1 + value: 8.166822398413025 + - type: nauc_recall_at_1000_max + value: 14.844192671123944 + - type: nauc_recall_at_1000_std + value: 19.89066147510425 + - type: nauc_recall_at_100_diff1 + value: 17.771335801647908 + - type: nauc_recall_at_100_max + value: 14.028210814051386 + - type: nauc_recall_at_100_std + value: 0.8854257983597732 + - type: nauc_recall_at_10_diff1 + value: 25.583966333098534 + - type: nauc_recall_at_10_max + value: 18.78312107431364 + - type: nauc_recall_at_10_std + value: -6.259396644779273 + - type: nauc_recall_at_1_diff1 + value: 46.880104967892095 + - type: nauc_recall_at_1_max + value: 30.47076747044864 + - type: nauc_recall_at_1_std + value: -9.529043223192849 + - type: nauc_recall_at_20_diff1 + value: 25.363665370206114 + - type: nauc_recall_at_20_max + value: 18.59288202265051 + - type: nauc_recall_at_20_std + value: -5.518742270755405 + - type: nauc_recall_at_3_diff1 + value: 33.02581275727718 + - type: nauc_recall_at_3_max + value: 24.221221318509823 + - type: nauc_recall_at_3_std + value: -8.806162684655048 + - type: nauc_recall_at_5_diff1 + value: 29.392786856617924 + - type: nauc_recall_at_5_max + value: 21.258229422853034 + - type: nauc_recall_at_5_std + value: -8.60839655648439 + - type: ndcg_at_1 + value: 12.592999999999998 + - type: ndcg_at_10 + value: 17.681 + - type: ndcg_at_100 + value: 21.708 + - type: ndcg_at_1000 + value: 25.224000000000004 + - type: ndcg_at_20 + value: 19.041 + - type: ndcg_at_3 + value: 15.101999999999999 + - type: ndcg_at_5 + value: 16.339000000000002 + - type: precision_at_1 + value: 12.592999999999998 + - type: precision_at_10 + value: 3.088 + - type: precision_at_100 + value: 0.567 + - type: precision_at_1000 + value: 0.098 + - type: precision_at_20 + value: 1.894 + - type: precision_at_3 + value: 7.058000000000001 + - type: precision_at_5 + value: 5.037 + - type: recall_at_1 + value: 10.693 + - type: recall_at_10 + value: 24.004 + - type: recall_at_100 + value: 42.614999999999995 + - type: recall_at_1000 + value: 68.407 + - type: recall_at_20 + value: 28.932000000000002 + - type: recall_at_3 + value: 16.892 + - type: recall_at_5 + value: 20.104 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackWebmastersRetrieval (default) + revision: 160c094312a0e1facb97e55eeddb698c0abe3571 + split: test + type: mteb/cqadupstack-webmasters + metrics: + - type: main_score + value: 21.083 + - type: map_at_1 + value: 10.589 + - type: map_at_10 + value: 16.74 + - type: map_at_100 + value: 17.746000000000002 + - type: map_at_1000 + value: 17.944 + - type: map_at_20 + value: 17.216 + - type: map_at_3 + value: 14.515 + - type: map_at_5 + value: 15.616 + - type: mrr_at_1 + value: 13.83399209486166 + - type: mrr_at_10 + value: 20.246251333207848 + - type: mrr_at_100 + value: 21.181360197232546 + - type: mrr_at_1000 + value: 21.275281349122263 + - type: mrr_at_20 + value: 20.722425203153367 + - type: mrr_at_3 + value: 18.28063241106719 + - type: mrr_at_5 + value: 19.308300395256918 + - type: nauc_map_at_1000_diff1 + value: 35.23000631718578 + - type: nauc_map_at_1000_max + value: 14.034629155313908 + - type: nauc_map_at_1000_std + value: -3.342132884415706 + - type: nauc_map_at_100_diff1 + value: 35.07001876750455 + - type: nauc_map_at_100_max + value: 14.080660368793952 + - type: nauc_map_at_100_std + value: -3.38242131987473 + - type: nauc_map_at_10_diff1 + value: 35.062025807984 + - type: nauc_map_at_10_max + value: 13.699511234621395 + - type: nauc_map_at_10_std + value: -4.083990022755827 + - type: nauc_map_at_1_diff1 + value: 45.349178620901554 + - type: nauc_map_at_1_max + value: 16.3746817879971 + - type: nauc_map_at_1_std + value: -9.540072950708476 + - type: nauc_map_at_20_diff1 + value: 34.98391565692215 + - type: nauc_map_at_20_max + value: 13.931709612931071 + - type: nauc_map_at_20_std + value: -3.6983773997754374 + - type: nauc_map_at_3_diff1 + value: 36.436904397354326 + - type: nauc_map_at_3_max + value: 14.045828808394539 + - type: nauc_map_at_3_std + value: -4.983728097939691 + - type: nauc_map_at_5_diff1 + value: 36.25507710491795 + - type: nauc_map_at_5_max + value: 13.519612188865631 + - type: nauc_map_at_5_std + value: -4.818257745540146 + - type: nauc_mrr_at_1000_diff1 + value: 36.944178495564515 + - type: nauc_mrr_at_1000_max + value: 14.749051712948209 + - type: nauc_mrr_at_1000_std + value: -1.370198226412231 + - type: nauc_mrr_at_100_diff1 + value: 36.92276956055459 + - type: nauc_mrr_at_100_max + value: 14.686792299720352 + - type: nauc_mrr_at_100_std + value: -1.410279466082495 + - type: nauc_mrr_at_10_diff1 + value: 37.201959911444554 + - type: nauc_mrr_at_10_max + value: 14.25820720715842 + - type: nauc_mrr_at_10_std + value: -1.7923078894976436 + - type: nauc_mrr_at_1_diff1 + value: 45.755628967932196 + - type: nauc_mrr_at_1_max + value: 16.607201054295086 + - type: nauc_mrr_at_1_std + value: -7.0573272764900965 + - type: nauc_mrr_at_20_diff1 + value: 36.95792032825288 + - type: nauc_mrr_at_20_max + value: 14.552930433669317 + - type: nauc_mrr_at_20_std + value: -1.543525246181849 + - type: nauc_mrr_at_3_diff1 + value: 38.61392452473216 + - type: nauc_mrr_at_3_max + value: 15.09000017892547 + - type: nauc_mrr_at_3_std + value: -2.4206532001081893 + - type: nauc_mrr_at_5_diff1 + value: 37.8928671772299 + - type: nauc_mrr_at_5_max + value: 14.007056266446194 + - type: nauc_mrr_at_5_std + value: -2.8039294686296974 + - type: nauc_ndcg_at_1000_diff1 + value: 33.17069478746769 + - type: nauc_ndcg_at_1000_max + value: 16.83474260617637 + - type: nauc_ndcg_at_1000_std + value: 2.526704315250276 + - type: nauc_ndcg_at_100_diff1 + value: 31.913331532987343 + - type: nauc_ndcg_at_100_max + value: 15.589987165503793 + - type: nauc_ndcg_at_100_std + value: 1.4127265220523926 + - type: nauc_ndcg_at_10_diff1 + value: 32.72185313773176 + - type: nauc_ndcg_at_10_max + value: 12.836797550950171 + - type: nauc_ndcg_at_10_std + value: -1.383053414866171 + - type: nauc_ndcg_at_1_diff1 + value: 45.755628967932196 + - type: nauc_ndcg_at_1_max + value: 16.607201054295086 + - type: nauc_ndcg_at_1_std + value: -7.0573272764900965 + - type: nauc_ndcg_at_20_diff1 + value: 32.210491715945096 + - type: nauc_ndcg_at_20_max + value: 13.854824191729895 + - type: nauc_ndcg_at_20_std + value: -0.32877605104736113 + - type: nauc_ndcg_at_3_diff1 + value: 35.77867047873611 + - type: nauc_ndcg_at_3_max + value: 13.436656917654846 + - type: nauc_ndcg_at_3_std + value: -2.2287189056300227 + - type: nauc_ndcg_at_5_diff1 + value: 34.88057269102703 + - type: nauc_ndcg_at_5_max + value: 12.012147587547142 + - type: nauc_ndcg_at_5_std + value: -2.912715446298086 + - type: nauc_precision_at_1000_diff1 + value: 18.52112163492622 + - type: nauc_precision_at_1000_max + value: 2.8134846288870183 + - type: nauc_precision_at_1000_std + value: 5.016718220137224 + - type: nauc_precision_at_100_diff1 + value: 24.36563458826747 + - type: nauc_precision_at_100_max + value: 5.139325190644362 + - type: nauc_precision_at_100_std + value: 5.411737126737265 + - type: nauc_precision_at_10_diff1 + value: 25.064506671188607 + - type: nauc_precision_at_10_max + value: 9.07537853265786 + - type: nauc_precision_at_10_std + value: 3.3546079369473714 + - type: nauc_precision_at_1_diff1 + value: 45.755628967932196 + - type: nauc_precision_at_1_max + value: 16.607201054295086 + - type: nauc_precision_at_1_std + value: -7.0573272764900965 + - type: nauc_precision_at_20_diff1 + value: 26.544768972255444 + - type: nauc_precision_at_20_max + value: 8.11877633227019 + - type: nauc_precision_at_20_std + value: 3.4908791358741156 + - type: nauc_precision_at_3_diff1 + value: 32.491133814451864 + - type: nauc_precision_at_3_max + value: 11.967165947670138 + - type: nauc_precision_at_3_std + value: 3.401586036732758 + - type: nauc_precision_at_5_diff1 + value: 30.046705015207348 + - type: nauc_precision_at_5_max + value: 7.852069800334351 + - type: nauc_precision_at_5_std + value: 0.691988259732557 + - type: nauc_recall_at_1000_diff1 + value: 20.54143072751401 + - type: nauc_recall_at_1000_max + value: 36.86342351337889 + - type: nauc_recall_at_1000_std + value: 29.782267794890714 + - type: nauc_recall_at_100_diff1 + value: 18.86293596621599 + - type: nauc_recall_at_100_max + value: 20.773347136028207 + - type: nauc_recall_at_100_std + value: 13.060795971533448 + - type: nauc_recall_at_10_diff1 + value: 23.567799982717375 + - type: nauc_recall_at_10_max + value: 11.114908013016226 + - type: nauc_recall_at_10_std + value: 3.185236554617246 + - type: nauc_recall_at_1_diff1 + value: 45.349178620901554 + - type: nauc_recall_at_1_max + value: 16.3746817879971 + - type: nauc_recall_at_1_std + value: -9.540072950708476 + - type: nauc_recall_at_20_diff1 + value: 21.79299761755684 + - type: nauc_recall_at_20_max + value: 13.92842132191322 + - type: nauc_recall_at_20_std + value: 5.828656933839459 + - type: nauc_recall_at_3_diff1 + value: 29.366971638804095 + - type: nauc_recall_at_3_max + value: 11.688109536903807 + - type: nauc_recall_at_3_std + value: -0.3524788019775435 + - type: nauc_recall_at_5_diff1 + value: 28.44796720594458 + - type: nauc_recall_at_5_max + value: 9.24949698451435 + - type: nauc_recall_at_5_std + value: -0.5686547673804909 + - type: ndcg_at_1 + value: 13.834 + - type: ndcg_at_10 + value: 21.083 + - type: ndcg_at_100 + value: 26.019 + - type: ndcg_at_1000 + value: 29.578 + - type: ndcg_at_20 + value: 22.63 + - type: ndcg_at_3 + value: 17.309 + - type: ndcg_at_5 + value: 18.802 + - type: precision_at_1 + value: 13.834 + - type: precision_at_10 + value: 4.387 + - type: precision_at_100 + value: 0.984 + - type: precision_at_1000 + value: 0.189 + - type: precision_at_20 + value: 2.787 + - type: precision_at_3 + value: 8.564 + - type: precision_at_5 + value: 6.4430000000000005 + - type: recall_at_1 + value: 10.589 + - type: recall_at_10 + value: 30.162 + - type: recall_at_100 + value: 53.418 + - type: recall_at_1000 + value: 77.039 + - type: recall_at_20 + value: 36.278 + - type: recall_at_3 + value: 18.917 + - type: recall_at_5 + value: 23.026 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackWordpressRetrieval (default) + revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4 + split: test + type: mteb/cqadupstack-wordpress + metrics: + - type: main_score + value: 14.978 + - type: map_at_1 + value: 8.725 + - type: map_at_10 + value: 12.397 + - type: map_at_100 + value: 13.200999999999999 + - type: map_at_1000 + value: 13.320000000000002 + - type: map_at_20 + value: 12.824 + - type: map_at_3 + value: 10.886 + - type: map_at_5 + value: 11.806999999999999 + - type: mrr_at_1 + value: 9.426987060998151 + - type: mrr_at_10 + value: 13.470058386879085 + - type: mrr_at_100 + value: 14.288265285643698 + - type: mrr_at_1000 + value: 14.388800653131446 + - type: mrr_at_20 + value: 13.905041096945778 + - type: mrr_at_3 + value: 11.76833025261861 + - type: mrr_at_5 + value: 12.87738755391251 + - type: nauc_map_at_1000_diff1 + value: 26.516673602508238 + - type: nauc_map_at_1000_max + value: 18.128898744784223 + - type: nauc_map_at_1000_std + value: -3.1241922128008364 + - type: nauc_map_at_100_diff1 + value: 26.529424903965506 + - type: nauc_map_at_100_max + value: 18.121027204975082 + - type: nauc_map_at_100_std + value: -3.2676259690788925 + - type: nauc_map_at_10_diff1 + value: 26.891005808479278 + - type: nauc_map_at_10_max + value: 18.150121635095932 + - type: nauc_map_at_10_std + value: -3.1818763691131964 + - type: nauc_map_at_1_diff1 + value: 34.52482250092936 + - type: nauc_map_at_1_max + value: 21.403211658773166 + - type: nauc_map_at_1_std + value: -7.354875662027327 + - type: nauc_map_at_20_diff1 + value: 26.71979132749755 + - type: nauc_map_at_20_max + value: 18.108875183007527 + - type: nauc_map_at_20_std + value: -3.4006672935922633 + - type: nauc_map_at_3_diff1 + value: 30.46429907650115 + - type: nauc_map_at_3_max + value: 20.327713395843396 + - type: nauc_map_at_3_std + value: -3.4148598410901316 + - type: nauc_map_at_5_diff1 + value: 27.98053127124393 + - type: nauc_map_at_5_max + value: 18.79633171976155 + - type: nauc_map_at_5_std + value: -3.1672774447353413 + - type: nauc_mrr_at_1000_diff1 + value: 27.329698975753857 + - type: nauc_mrr_at_1000_max + value: 19.13329896851427 + - type: nauc_mrr_at_1000_std + value: -2.6699259566824516 + - type: nauc_mrr_at_100_diff1 + value: 27.342939797507377 + - type: nauc_mrr_at_100_max + value: 19.12292516908982 + - type: nauc_mrr_at_100_std + value: -2.7630089030531506 + - type: nauc_mrr_at_10_diff1 + value: 27.70674345186378 + - type: nauc_mrr_at_10_max + value: 19.1410332139274 + - type: nauc_mrr_at_10_std + value: -2.658242192942558 + - type: nauc_mrr_at_1_diff1 + value: 35.415242782938975 + - type: nauc_mrr_at_1_max + value: 22.701431127854693 + - type: nauc_mrr_at_1_std + value: -6.788407573683776 + - type: nauc_mrr_at_20_diff1 + value: 27.525159054745714 + - type: nauc_mrr_at_20_max + value: 19.072297716380486 + - type: nauc_mrr_at_20_std + value: -2.873908685324186 + - type: nauc_mrr_at_3_diff1 + value: 31.272337691671826 + - type: nauc_mrr_at_3_max + value: 21.135406794969285 + - type: nauc_mrr_at_3_std + value: -3.4310898471972044 + - type: nauc_mrr_at_5_diff1 + value: 28.663498942865967 + - type: nauc_mrr_at_5_max + value: 19.869337721787417 + - type: nauc_mrr_at_5_std + value: -2.7477312597184036 + - type: nauc_ndcg_at_1000_diff1 + value: 21.50039886741295 + - type: nauc_ndcg_at_1000_max + value: 16.76172996905625 + - type: nauc_ndcg_at_1000_std + value: 1.0698201743855864 + - type: nauc_ndcg_at_100_diff1 + value: 21.629922756878774 + - type: nauc_ndcg_at_100_max + value: 16.404192109073197 + - type: nauc_ndcg_at_100_std + value: -1.5166564388971011 + - type: nauc_ndcg_at_10_diff1 + value: 23.368742832143244 + - type: nauc_ndcg_at_10_max + value: 16.329548177009606 + - type: nauc_ndcg_at_10_std + value: -2.1458488442168235 + - type: nauc_ndcg_at_1_diff1 + value: 35.415242782938975 + - type: nauc_ndcg_at_1_max + value: 22.701431127854693 + - type: nauc_ndcg_at_1_std + value: -6.788407573683776 + - type: nauc_ndcg_at_20_diff1 + value: 22.793508520783316 + - type: nauc_ndcg_at_20_max + value: 16.05112254209437 + - type: nauc_ndcg_at_20_std + value: -2.761873160653705 + - type: nauc_ndcg_at_3_diff1 + value: 29.82956298838037 + - type: nauc_ndcg_at_3_max + value: 20.062474567529048 + - type: nauc_ndcg_at_3_std + value: -2.4980769789508668 + - type: nauc_ndcg_at_5_diff1 + value: 25.66424410602312 + - type: nauc_ndcg_at_5_max + value: 17.601576278936605 + - type: nauc_ndcg_at_5_std + value: -2.3231890985166603 + - type: nauc_precision_at_1000_diff1 + value: 3.709316535367016 + - type: nauc_precision_at_1000_max + value: 10.546640188020135 + - type: nauc_precision_at_1000_std + value: 18.202854906721562 + - type: nauc_precision_at_100_diff1 + value: 8.291596389411799 + - type: nauc_precision_at_100_max + value: 14.644356710423942 + - type: nauc_precision_at_100_std + value: 5.259044565744187 + - type: nauc_precision_at_10_diff1 + value: 14.9863600560976 + - type: nauc_precision_at_10_max + value: 12.772849786137709 + - type: nauc_precision_at_10_std + value: 1.6621034698073962 + - type: nauc_precision_at_1_diff1 + value: 35.415242782938975 + - type: nauc_precision_at_1_max + value: 22.701431127854693 + - type: nauc_precision_at_1_std + value: -6.788407573683776 + - type: nauc_precision_at_20_diff1 + value: 13.084814659316926 + - type: nauc_precision_at_20_max + value: 13.024432776766675 + - type: nauc_precision_at_20_std + value: 0.18909201285442837 + - type: nauc_precision_at_3_diff1 + value: 27.936920978190987 + - type: nauc_precision_at_3_max + value: 18.680011829390438 + - type: nauc_precision_at_3_std + value: -1.1851787205474336 + - type: nauc_precision_at_5_diff1 + value: 20.487312379443644 + - type: nauc_precision_at_5_max + value: 15.500096864522916 + - type: nauc_precision_at_5_std + value: 0.20075767744041675 + - type: nauc_recall_at_1000_diff1 + value: 5.762022210382103 + - type: nauc_recall_at_1000_max + value: 13.160918374309539 + - type: nauc_recall_at_1000_std + value: 19.14372005911137 + - type: nauc_recall_at_100_diff1 + value: 10.314461398699912 + - type: nauc_recall_at_100_max + value: 11.95089820438129 + - type: nauc_recall_at_100_std + value: 2.5226003737726694 + - type: nauc_recall_at_10_diff1 + value: 14.809747725503208 + - type: nauc_recall_at_10_max + value: 11.374320213352947 + - type: nauc_recall_at_10_std + value: -0.5456850002607359 + - type: nauc_recall_at_1_diff1 + value: 34.52482250092936 + - type: nauc_recall_at_1_max + value: 21.403211658773166 + - type: nauc_recall_at_1_std + value: -7.354875662027327 + - type: nauc_recall_at_20_diff1 + value: 14.093871737295386 + - type: nauc_recall_at_20_max + value: 10.673857337966515 + - type: nauc_recall_at_20_std + value: -2.282867759773375 + - type: nauc_recall_at_3_diff1 + value: 26.908608208530637 + - type: nauc_recall_at_3_max + value: 18.078128413017662 + - type: nauc_recall_at_3_std + value: -0.7517685230312128 + - type: nauc_recall_at_5_diff1 + value: 19.854119160897675 + - type: nauc_recall_at_5_max + value: 13.986759331769022 + - type: nauc_recall_at_5_std + value: -0.724937240578743 + - type: ndcg_at_1 + value: 9.427000000000001 + - type: ndcg_at_10 + value: 14.978 + - type: ndcg_at_100 + value: 19.341 + - type: ndcg_at_1000 + value: 22.834 + - type: ndcg_at_20 + value: 16.483999999999998 + - type: ndcg_at_3 + value: 11.875 + - type: ndcg_at_5 + value: 13.536999999999999 + - type: precision_at_1 + value: 9.427000000000001 + - type: precision_at_10 + value: 2.495 + - type: precision_at_100 + value: 0.508 + - type: precision_at_1000 + value: 0.086 + - type: precision_at_20 + value: 1.608 + - type: precision_at_3 + value: 4.9910000000000005 + - type: precision_at_5 + value: 3.9190000000000005 + - type: recall_at_1 + value: 8.725 + - type: recall_at_10 + value: 22.020999999999997 + - type: recall_at_100 + value: 42.585 + - type: recall_at_1000 + value: 69.87400000000001 + - type: recall_at_20 + value: 27.572000000000003 + - type: recall_at_3 + value: 13.869000000000002 + - type: recall_at_5 + value: 17.702 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ClimateFEVER (default) + revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380 + split: test + type: mteb/climate-fever + metrics: + - type: main_score + value: 14.771999999999998 + - type: map_at_1 + value: 5.5489999999999995 + - type: map_at_10 + value: 9.677 + - type: map_at_100 + value: 10.946 + - type: map_at_1000 + value: 11.119 + - type: map_at_20 + value: 10.335999999999999 + - type: map_at_3 + value: 7.869 + - type: map_at_5 + value: 8.741999999999999 + - type: mrr_at_1 + value: 13.029315960912053 + - type: mrr_at_10 + value: 20.339744584044244 + - type: mrr_at_100 + value: 21.54484002453409 + - type: mrr_at_1000 + value: 21.616919381723786 + - type: mrr_at_20 + value: 21.139241607046472 + - type: mrr_at_3 + value: 17.491856677524435 + - type: mrr_at_5 + value: 19.071661237785012 + - type: nauc_map_at_1000_diff1 + value: 26.164411068020073 + - type: nauc_map_at_1000_max + value: 23.58735724160517 + - type: nauc_map_at_1000_std + value: 18.779016401499614 + - type: nauc_map_at_100_diff1 + value: 26.214988804799972 + - type: nauc_map_at_100_max + value: 23.406310727713407 + - type: nauc_map_at_100_std + value: 18.460043360320917 + - type: nauc_map_at_10_diff1 + value: 26.718397914143065 + - type: nauc_map_at_10_max + value: 21.229694457847526 + - type: nauc_map_at_10_std + value: 15.390083455519378 + - type: nauc_map_at_1_diff1 + value: 37.32766813952627 + - type: nauc_map_at_1_max + value: 19.515906010846315 + - type: nauc_map_at_1_std + value: 9.99207652106678 + - type: nauc_map_at_20_diff1 + value: 26.465578269196715 + - type: nauc_map_at_20_max + value: 22.398156286792933 + - type: nauc_map_at_20_std + value: 17.078418627633006 + - type: nauc_map_at_3_diff1 + value: 29.759256354844837 + - type: nauc_map_at_3_max + value: 18.385599223153708 + - type: nauc_map_at_3_std + value: 11.366056030872816 + - type: nauc_map_at_5_diff1 + value: 28.241520126871393 + - type: nauc_map_at_5_max + value: 19.78438163465289 + - type: nauc_map_at_5_std + value: 13.459940207726998 + - type: nauc_mrr_at_1000_diff1 + value: 21.12409148907605 + - type: nauc_mrr_at_1000_max + value: 23.77746871154926 + - type: nauc_mrr_at_1000_std + value: 20.15537425469562 + - type: nauc_mrr_at_100_diff1 + value: 21.12004667990828 + - type: nauc_mrr_at_100_max + value: 23.78827313202278 + - type: nauc_mrr_at_100_std + value: 20.187546696894692 + - type: nauc_mrr_at_10_diff1 + value: 21.183801786810566 + - type: nauc_mrr_at_10_max + value: 23.02472680238422 + - type: nauc_mrr_at_10_std + value: 18.94062182333125 + - type: nauc_mrr_at_1_diff1 + value: 29.250683054246483 + - type: nauc_mrr_at_1_max + value: 21.216233159084112 + - type: nauc_mrr_at_1_std + value: 13.676910696549449 + - type: nauc_mrr_at_20_diff1 + value: 21.038599683577484 + - type: nauc_mrr_at_20_max + value: 23.712981135036166 + - type: nauc_mrr_at_20_std + value: 20.008317056296054 + - type: nauc_mrr_at_3_diff1 + value: 21.924062586657143 + - type: nauc_mrr_at_3_max + value: 21.42184982676593 + - type: nauc_mrr_at_3_std + value: 16.457179657031556 + - type: nauc_mrr_at_5_diff1 + value: 21.656696900775223 + - type: nauc_mrr_at_5_max + value: 21.98298849500632 + - type: nauc_mrr_at_5_std + value: 17.601656923653913 + - type: nauc_ndcg_at_1000_diff1 + value: 19.858622397094305 + - type: nauc_ndcg_at_1000_max + value: 30.747733801747284 + - type: nauc_ndcg_at_1000_std + value: 31.099839929023442 + - type: nauc_ndcg_at_100_diff1 + value: 20.47505630498792 + - type: nauc_ndcg_at_100_max + value: 29.271746216110035 + - type: nauc_ndcg_at_100_std + value: 28.396186616156115 + - type: nauc_ndcg_at_10_diff1 + value: 21.563095586027515 + - type: nauc_ndcg_at_10_max + value: 23.47146695177757 + - type: nauc_ndcg_at_10_std + value: 19.743756127255523 + - type: nauc_ndcg_at_1_diff1 + value: 29.250683054246483 + - type: nauc_ndcg_at_1_max + value: 21.216233159084112 + - type: nauc_ndcg_at_1_std + value: 13.676910696549449 + - type: nauc_ndcg_at_20_diff1 + value: 20.85499472041572 + - type: nauc_ndcg_at_20_max + value: 25.959345574497316 + - type: nauc_ndcg_at_20_std + value: 23.727313893098067 + - type: nauc_ndcg_at_3_diff1 + value: 24.777898795425166 + - type: nauc_ndcg_at_3_max + value: 20.273718923990444 + - type: nauc_ndcg_at_3_std + value: 14.474447188126685 + - type: nauc_ndcg_at_5_diff1 + value: 23.66758409019224 + - type: nauc_ndcg_at_5_max + value: 20.977206695913566 + - type: nauc_ndcg_at_5_std + value: 16.27106724405967 + - type: nauc_precision_at_1000_diff1 + value: 0.8707948826157808 + - type: nauc_precision_at_1000_max + value: 34.30239711148936 + - type: nauc_precision_at_1000_std + value: 43.664633287089586 + - type: nauc_precision_at_100_diff1 + value: 6.854023820044692 + - type: nauc_precision_at_100_max + value: 37.282323173227546 + - type: nauc_precision_at_100_std + value: 42.33392649552268 + - type: nauc_precision_at_10_diff1 + value: 11.091638888631259 + - type: nauc_precision_at_10_max + value: 28.594826391973132 + - type: nauc_precision_at_10_std + value: 29.321648755228264 + - type: nauc_precision_at_1_diff1 + value: 29.250683054246483 + - type: nauc_precision_at_1_max + value: 21.216233159084112 + - type: nauc_precision_at_1_std + value: 13.676910696549449 + - type: nauc_precision_at_20_diff1 + value: 9.594100825895916 + - type: nauc_precision_at_20_max + value: 33.200054445724255 + - type: nauc_precision_at_20_std + value: 36.78985295189952 + - type: nauc_precision_at_3_diff1 + value: 17.110916647434916 + - type: nauc_precision_at_3_max + value: 21.20074776254172 + - type: nauc_precision_at_3_std + value: 18.67703775540859 + - type: nauc_precision_at_5_diff1 + value: 15.501350898541641 + - type: nauc_precision_at_5_max + value: 24.499520173206353 + - type: nauc_precision_at_5_std + value: 23.53022513283104 + - type: nauc_recall_at_1000_diff1 + value: 7.210483259093574 + - type: nauc_recall_at_1000_max + value: 35.51790397302799 + - type: nauc_recall_at_1000_std + value: 45.97048584755478 + - type: nauc_recall_at_100_diff1 + value: 11.112587178551932 + - type: nauc_recall_at_100_max + value: 31.283221366595114 + - type: nauc_recall_at_100_std + value: 36.8933237158813 + - type: nauc_recall_at_10_diff1 + value: 14.523694347361332 + - type: nauc_recall_at_10_max + value: 22.980982127733885 + - type: nauc_recall_at_10_std + value: 21.638404151421625 + - type: nauc_recall_at_1_diff1 + value: 37.32766813952627 + - type: nauc_recall_at_1_max + value: 19.515906010846315 + - type: nauc_recall_at_1_std + value: 9.99207652106678 + - type: nauc_recall_at_20_diff1 + value: 12.344344516597259 + - type: nauc_recall_at_20_max + value: 26.432279445759914 + - type: nauc_recall_at_20_std + value: 28.612814617982608 + - type: nauc_recall_at_3_diff1 + value: 22.559118848361635 + - type: nauc_recall_at_3_max + value: 17.644261245104172 + - type: nauc_recall_at_3_std + value: 12.654124975909692 + - type: nauc_recall_at_5_diff1 + value: 19.208115562541288 + - type: nauc_recall_at_5_max + value: 19.766812655523903 + - type: nauc_recall_at_5_std + value: 16.378285780040738 + - type: ndcg_at_1 + value: 13.028999999999998 + - type: ndcg_at_10 + value: 14.771999999999998 + - type: ndcg_at_100 + value: 20.779 + - type: ndcg_at_1000 + value: 24.553 + - type: ndcg_at_20 + value: 17.085 + - type: ndcg_at_3 + value: 11.203000000000001 + - type: ndcg_at_5 + value: 12.475 + - type: precision_at_1 + value: 13.028999999999998 + - type: precision_at_10 + value: 4.893 + - type: precision_at_100 + value: 1.126 + - type: precision_at_1000 + value: 0.181 + - type: precision_at_20 + value: 3.427 + - type: precision_at_3 + value: 8.578 + - type: precision_at_5 + value: 6.9190000000000005 + - type: recall_at_1 + value: 5.5489999999999995 + - type: recall_at_10 + value: 18.955 + - type: recall_at_100 + value: 40.041 + - type: recall_at_1000 + value: 61.970000000000006 + - type: recall_at_20 + value: 25.535999999999998 + - type: recall_at_3 + value: 10.41 + - type: recall_at_5 + value: 13.636999999999999 + task: + type: Retrieval + - dataset: + config: default + name: MTEB DBPedia (default) + revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659 + split: test + type: mteb/dbpedia + metrics: + - type: main_score + value: 22.325999999999997 + - type: map_at_1 + value: 4.059 + - type: map_at_10 + value: 8.817 + - type: map_at_100 + value: 12.668 + - type: map_at_1000 + value: 13.559 + - type: map_at_20 + value: 10.344000000000001 + - type: map_at_3 + value: 6.4030000000000005 + - type: map_at_5 + value: 7.411 + - type: mrr_at_1 + value: 41.0 + - type: mrr_at_10 + value: 50.98363095238096 + - type: mrr_at_100 + value: 51.72977669775032 + - type: mrr_at_1000 + value: 51.752327140423695 + - type: mrr_at_20 + value: 51.50770148324331 + - type: mrr_at_3 + value: 48.0 + - type: mrr_at_5 + value: 49.5125 + - type: nauc_map_at_1000_diff1 + value: 29.149373818439265 + - type: nauc_map_at_1000_max + value: 28.195342365031212 + - type: nauc_map_at_1000_std + value: 35.55065720761848 + - type: nauc_map_at_100_diff1 + value: 29.348983394865396 + - type: nauc_map_at_100_max + value: 25.633550032614806 + - type: nauc_map_at_100_std + value: 32.52451720021242 + - type: nauc_map_at_10_diff1 + value: 36.25237616876193 + - type: nauc_map_at_10_max + value: 10.433718867684474 + - type: nauc_map_at_10_std + value: 16.315904343848334 + - type: nauc_map_at_1_diff1 + value: 48.50316629206164 + - type: nauc_map_at_1_max + value: 2.5571639528096277 + - type: nauc_map_at_1_std + value: 5.472788157421047 + - type: nauc_map_at_20_diff1 + value: 34.45541065707478 + - type: nauc_map_at_20_max + value: 15.48733575140639 + - type: nauc_map_at_20_std + value: 22.375360952444645 + - type: nauc_map_at_3_diff1 + value: 40.58349942532284 + - type: nauc_map_at_3_max + value: 4.194993964973337 + - type: nauc_map_at_3_std + value: 8.737777513681708 + - type: nauc_map_at_5_diff1 + value: 39.0798474228746 + - type: nauc_map_at_5_max + value: 5.429548462682357 + - type: nauc_map_at_5_std + value: 10.687290975485238 + - type: nauc_mrr_at_1000_diff1 + value: 31.26810228227898 + - type: nauc_mrr_at_1000_max + value: 43.28972948203106 + - type: nauc_mrr_at_1000_std + value: 27.97866115458008 + - type: nauc_mrr_at_100_diff1 + value: 31.252414298595298 + - type: nauc_mrr_at_100_max + value: 43.28429517353382 + - type: nauc_mrr_at_100_std + value: 27.981652093220422 + - type: nauc_mrr_at_10_diff1 + value: 31.31952045872999 + - type: nauc_mrr_at_10_max + value: 43.15007965554428 + - type: nauc_mrr_at_10_std + value: 27.54085208273479 + - type: nauc_mrr_at_1_diff1 + value: 33.447111099629296 + - type: nauc_mrr_at_1_max + value: 40.639975715928024 + - type: nauc_mrr_at_1_std + value: 27.664466460854854 + - type: nauc_mrr_at_20_diff1 + value: 31.29766674009583 + - type: nauc_mrr_at_20_max + value: 43.22994957875351 + - type: nauc_mrr_at_20_std + value: 27.9022380444922 + - type: nauc_mrr_at_3_diff1 + value: 31.456550406372802 + - type: nauc_mrr_at_3_max + value: 43.45211663706338 + - type: nauc_mrr_at_3_std + value: 27.754622154044906 + - type: nauc_mrr_at_5_diff1 + value: 31.744167906758573 + - type: nauc_mrr_at_5_max + value: 43.358567652212976 + - type: nauc_mrr_at_5_std + value: 27.95435203187726 + - type: nauc_ndcg_at_1000_diff1 + value: 28.872533419650097 + - type: nauc_ndcg_at_1000_max + value: 38.367438067028445 + - type: nauc_ndcg_at_1000_std + value: 45.56262810614998 + - type: nauc_ndcg_at_100_diff1 + value: 28.224480561200004 + - type: nauc_ndcg_at_100_max + value: 30.269004490486186 + - type: nauc_ndcg_at_100_std + value: 36.680948795273444 + - type: nauc_ndcg_at_10_diff1 + value: 30.11778029333171 + - type: nauc_ndcg_at_10_max + value: 29.25662355248585 + - type: nauc_ndcg_at_10_std + value: 28.37564750887575 + - type: nauc_ndcg_at_1_diff1 + value: 34.200585955057214 + - type: nauc_ndcg_at_1_max + value: 27.32267704776983 + - type: nauc_ndcg_at_1_std + value: 20.070336621457255 + - type: nauc_ndcg_at_20_diff1 + value: 30.156546268588553 + - type: nauc_ndcg_at_20_max + value: 25.99567370220745 + - type: nauc_ndcg_at_20_std + value: 29.36968221162758 + - type: nauc_ndcg_at_3_diff1 + value: 29.642608252509245 + - type: nauc_ndcg_at_3_max + value: 31.964467916838306 + - type: nauc_ndcg_at_3_std + value: 25.549340163113797 + - type: nauc_ndcg_at_5_diff1 + value: 30.104992922460404 + - type: nauc_ndcg_at_5_max + value: 31.04270512814639 + - type: nauc_ndcg_at_5_std + value: 27.688390384767757 + - type: nauc_precision_at_1000_diff1 + value: -7.77879940579617 + - type: nauc_precision_at_1000_max + value: 29.02383046450357 + - type: nauc_precision_at_1000_std + value: 29.239802521893793 + - type: nauc_precision_at_100_diff1 + value: 0.5367752232887497 + - type: nauc_precision_at_100_max + value: 45.20878632602283 + - type: nauc_precision_at_100_std + value: 44.818028589063324 + - type: nauc_precision_at_10_diff1 + value: 12.42234237789027 + - type: nauc_precision_at_10_max + value: 43.260904263882104 + - type: nauc_precision_at_10_std + value: 38.33705651495131 + - type: nauc_precision_at_1_diff1 + value: 33.447111099629296 + - type: nauc_precision_at_1_max + value: 40.639975715928024 + - type: nauc_precision_at_1_std + value: 27.664466460854854 + - type: nauc_precision_at_20_diff1 + value: 8.665744338069603 + - type: nauc_precision_at_20_max + value: 45.42654329313483 + - type: nauc_precision_at_20_std + value: 43.01981922769551 + - type: nauc_precision_at_3_diff1 + value: 18.612027834320646 + - type: nauc_precision_at_3_max + value: 41.74529031915723 + - type: nauc_precision_at_3_std + value: 31.443614275628185 + - type: nauc_precision_at_5_diff1 + value: 16.59901783924638 + - type: nauc_precision_at_5_max + value: 43.87805345673531 + - type: nauc_precision_at_5_std + value: 35.77729700994364 + - type: nauc_recall_at_1000_diff1 + value: 15.605491669695336 + - type: nauc_recall_at_1000_max + value: 28.937032865375567 + - type: nauc_recall_at_1000_std + value: 51.29409263551372 + - type: nauc_recall_at_100_diff1 + value: 14.60626342855007 + - type: nauc_recall_at_100_max + value: 19.742808089206772 + - type: nauc_recall_at_100_std + value: 34.002728054787305 + - type: nauc_recall_at_10_diff1 + value: 23.608867385986805 + - type: nauc_recall_at_10_max + value: -1.5056012290538128 + - type: nauc_recall_at_10_std + value: 6.72388525739291 + - type: nauc_recall_at_1_diff1 + value: 48.50316629206164 + - type: nauc_recall_at_1_max + value: 2.5571639528096277 + - type: nauc_recall_at_1_std + value: 5.472788157421047 + - type: nauc_recall_at_20_diff1 + value: 23.169940060107784 + - type: nauc_recall_at_20_max + value: 3.3775837661388577 + - type: nauc_recall_at_20_std + value: 14.687180697705957 + - type: nauc_recall_at_3_diff1 + value: 34.02785181074422 + - type: nauc_recall_at_3_max + value: 0.16807738159490754 + - type: nauc_recall_at_3_std + value: 3.697714714085979 + - type: nauc_recall_at_5_diff1 + value: 29.72623580447505 + - type: nauc_recall_at_5_max + value: -3.2040020921850876 + - type: nauc_recall_at_5_std + value: 3.1413050324155187 + - type: ndcg_at_1 + value: 29.875 + - type: ndcg_at_10 + value: 22.325999999999997 + - type: ndcg_at_100 + value: 26.064999999999998 + - type: ndcg_at_1000 + value: 32.281 + - type: ndcg_at_20 + value: 22.470000000000002 + - type: ndcg_at_3 + value: 25.022 + - type: ndcg_at_5 + value: 23.202 + - type: precision_at_1 + value: 41.0 + - type: precision_at_10 + value: 19.875 + - type: precision_at_100 + value: 6.575 + - type: precision_at_1000 + value: 1.341 + - type: precision_at_20 + value: 15.475 + - type: precision_at_3 + value: 30.416999999999998 + - type: precision_at_5 + value: 24.9 + - type: recall_at_1 + value: 4.059 + - type: recall_at_10 + value: 13.783999999999999 + - type: recall_at_100 + value: 32.749 + - type: recall_at_1000 + value: 54.588 + - type: recall_at_20 + value: 19.334 + - type: recall_at_3 + value: 7.481999999999999 + - type: recall_at_5 + value: 9.869 + task: + type: Retrieval + - dataset: + config: default + name: MTEB EmotionClassification (default) + revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 + split: test + type: mteb/emotion + metrics: + - type: accuracy + value: 44.71 + - type: f1 + value: 40.4341929223557 + - type: f1_weighted + value: 46.99336148050091 + - type: main_score + value: 44.71 + task: + type: Classification + - dataset: + config: default + name: MTEB FEVER (default) + revision: bea83ef9e8fb933d90a2f1d5515737465d613e12 + split: test + type: mteb/fever + metrics: + - type: main_score + value: 26.815 + - type: map_at_1 + value: 14.388000000000002 + - type: map_at_10 + value: 21.918000000000003 + - type: map_at_100 + value: 22.965 + - type: map_at_1000 + value: 23.039 + - type: map_at_20 + value: 22.529 + - type: map_at_3 + value: 19.299 + - type: map_at_5 + value: 20.616 + - type: mrr_at_1 + value: 15.241524152415241 + - type: mrr_at_10 + value: 23.248259349744465 + - type: mrr_at_100 + value: 24.29525983878544 + - type: mrr_at_1000 + value: 24.364168282948302 + - type: mrr_at_20 + value: 23.8732428545265 + - type: mrr_at_3 + value: 20.48204820482046 + - type: mrr_at_5 + value: 21.883188318831934 + - type: nauc_map_at_1000_diff1 + value: 21.655721363953035 + - type: nauc_map_at_1000_max + value: 7.62632839503459 + - type: nauc_map_at_1000_std + value: -5.96263011077555 + - type: nauc_map_at_100_diff1 + value: 21.65449915845023 + - type: nauc_map_at_100_max + value: 7.620603349218483 + - type: nauc_map_at_100_std + value: -5.971649197942024 + - type: nauc_map_at_10_diff1 + value: 21.739985684584525 + - type: nauc_map_at_10_max + value: 7.088916079578887 + - type: nauc_map_at_10_std + value: -6.8118419036657665 + - type: nauc_map_at_1_diff1 + value: 27.903781721236314 + - type: nauc_map_at_1_max + value: 4.87951257147686 + - type: nauc_map_at_1_std + value: -9.402837727588127 + - type: nauc_map_at_20_diff1 + value: 21.667911401750317 + - type: nauc_map_at_20_max + value: 7.450247274464644 + - type: nauc_map_at_20_std + value: -6.28025758640821 + - type: nauc_map_at_3_diff1 + value: 23.068479934395373 + - type: nauc_map_at_3_max + value: 6.054163763350815 + - type: nauc_map_at_3_std + value: -7.968430487910183 + - type: nauc_map_at_5_diff1 + value: 22.003598455582406 + - type: nauc_map_at_5_max + value: 6.667068621095155 + - type: nauc_map_at_5_std + value: -7.356892532294304 + - type: nauc_mrr_at_1000_diff1 + value: 21.43312812034654 + - type: nauc_mrr_at_1000_max + value: 7.853432973026932 + - type: nauc_mrr_at_1000_std + value: -6.016594874669531 + - type: nauc_mrr_at_100_diff1 + value: 21.421027736461575 + - type: nauc_mrr_at_100_max + value: 7.857695018836121 + - type: nauc_mrr_at_100_std + value: -6.008152225210861 + - type: nauc_mrr_at_10_diff1 + value: 21.47275704705045 + - type: nauc_mrr_at_10_max + value: 7.361739773464361 + - type: nauc_mrr_at_10_std + value: -6.756627592543245 + - type: nauc_mrr_at_1_diff1 + value: 27.60421073842263 + - type: nauc_mrr_at_1_max + value: 4.998056898839529 + - type: nauc_mrr_at_1_std + value: -9.442871457148726 + - type: nauc_mrr_at_20_diff1 + value: 21.401638574498012 + - type: nauc_mrr_at_20_max + value: 7.689207705331705 + - type: nauc_mrr_at_20_std + value: -6.267461818245488 + - type: nauc_mrr_at_3_diff1 + value: 22.706324875653372 + - type: nauc_mrr_at_3_max + value: 6.227630142636092 + - type: nauc_mrr_at_3_std + value: -8.065029180110763 + - type: nauc_mrr_at_5_diff1 + value: 21.761811369171628 + - type: nauc_mrr_at_5_max + value: 6.9062748987713265 + - type: nauc_mrr_at_5_std + value: -7.3516871119086975 + - type: nauc_ndcg_at_1000_diff1 + value: 19.10514687283966 + - type: nauc_ndcg_at_1000_max + value: 10.896927197321096 + - type: nauc_ndcg_at_1000_std + value: -1.1781904676819126 + - type: nauc_ndcg_at_100_diff1 + value: 18.99144163290363 + - type: nauc_ndcg_at_100_max + value: 10.891493894654618 + - type: nauc_ndcg_at_100_std + value: -1.2375767691785773 + - type: nauc_ndcg_at_10_diff1 + value: 19.425260233591274 + - type: nauc_ndcg_at_10_max + value: 8.5170285343844 + - type: nauc_ndcg_at_10_std + value: -5.08694141032149 + - type: nauc_ndcg_at_1_diff1 + value: 27.60421073842263 + - type: nauc_ndcg_at_1_max + value: 4.998056898839529 + - type: nauc_ndcg_at_1_std + value: -9.442871457148726 + - type: nauc_ndcg_at_20_diff1 + value: 19.182333879372766 + - type: nauc_ndcg_at_20_max + value: 9.725326537318061 + - type: nauc_ndcg_at_20_std + value: -3.3215888170265337 + - type: nauc_ndcg_at_3_diff1 + value: 21.68892037139692 + - type: nauc_ndcg_at_3_max + value: 6.424582762507422 + - type: nauc_ndcg_at_3_std + value: -7.446689750696121 + - type: nauc_ndcg_at_5_diff1 + value: 19.98277916021606 + - type: nauc_ndcg_at_5_max + value: 7.503447105677768 + - type: nauc_ndcg_at_5_std + value: -6.353055635065993 + - type: nauc_precision_at_1000_diff1 + value: 0.94551504365766 + - type: nauc_precision_at_1000_max + value: 26.202950984563913 + - type: nauc_precision_at_1000_std + value: 26.01649052806106 + - type: nauc_precision_at_100_diff1 + value: 7.500434896405031 + - type: nauc_precision_at_100_max + value: 23.081873269296366 + - type: nauc_precision_at_100_std + value: 18.67428201659582 + - type: nauc_precision_at_10_diff1 + value: 13.347162393780213 + - type: nauc_precision_at_10_max + value: 12.824521844898237 + - type: nauc_precision_at_10_std + value: -0.3528264106373464 + - type: nauc_precision_at_1_diff1 + value: 27.60421073842263 + - type: nauc_precision_at_1_max + value: 4.998056898839529 + - type: nauc_precision_at_1_std + value: -9.442871457148726 + - type: nauc_precision_at_20_diff1 + value: 11.890778484289347 + - type: nauc_precision_at_20_max + value: 16.888519506633404 + - type: nauc_precision_at_20_std + value: 5.986698398244371 + - type: nauc_precision_at_3_diff1 + value: 18.076145969615585 + - type: nauc_precision_at_3_max + value: 7.3944113909218405 + - type: nauc_precision_at_3_std + value: -6.161877822968247 + - type: nauc_precision_at_5_diff1 + value: 14.735397341254108 + - type: nauc_precision_at_5_max + value: 9.743184544463226 + - type: nauc_precision_at_5_std + value: -3.8509248959327995 + - type: nauc_recall_at_1000_diff1 + value: 5.1388788747445036 + - type: nauc_recall_at_1000_max + value: 28.08149984627779 + - type: nauc_recall_at_1000_std + value: 29.07743185455911 + - type: nauc_recall_at_100_diff1 + value: 9.609814975019047 + - type: nauc_recall_at_100_max + value: 22.2426133779678 + - type: nauc_recall_at_100_std + value: 17.486934350968994 + - type: nauc_recall_at_10_diff1 + value: 13.521910300882013 + - type: nauc_recall_at_10_max + value: 11.50993372176311 + - type: nauc_recall_at_10_std + value: -0.6904522448973269 + - type: nauc_recall_at_1_diff1 + value: 27.903781721236314 + - type: nauc_recall_at_1_max + value: 4.87951257147686 + - type: nauc_recall_at_1_std + value: -9.402837727588127 + - type: nauc_recall_at_20_diff1 + value: 12.471865804084342 + - type: nauc_recall_at_20_max + value: 15.46145355412204 + - type: nauc_recall_at_20_std + value: 5.141306595627015 + - type: nauc_recall_at_3_diff1 + value: 18.546625288206567 + - type: nauc_recall_at_3_max + value: 7.1131838964541325 + - type: nauc_recall_at_3_std + value: -5.949003808218984 + - type: nauc_recall_at_5_diff1 + value: 14.902378618773394 + - type: nauc_recall_at_5_max + value: 9.104574501069981 + - type: nauc_recall_at_5_std + value: -3.7870817474786387 + - type: ndcg_at_1 + value: 15.242 + - type: ndcg_at_10 + value: 26.815 + - type: ndcg_at_100 + value: 32.104 + - type: ndcg_at_1000 + value: 34.182 + - type: ndcg_at_20 + value: 29.017 + - type: ndcg_at_3 + value: 21.255 + - type: ndcg_at_5 + value: 23.652 + - type: precision_at_1 + value: 15.242 + - type: precision_at_10 + value: 4.443 + - type: precision_at_100 + value: 0.7250000000000001 + - type: precision_at_1000 + value: 0.092 + - type: precision_at_20 + value: 2.696 + - type: precision_at_3 + value: 9.211 + - type: precision_at_5 + value: 6.796 + - type: recall_at_1 + value: 14.388000000000002 + - type: recall_at_10 + value: 41.082 + - type: recall_at_100 + value: 65.689 + - type: recall_at_1000 + value: 81.819 + - type: recall_at_20 + value: 49.55 + - type: recall_at_3 + value: 25.728 + - type: recall_at_5 + value: 31.5 + task: + type: Retrieval + - dataset: + config: default + name: MTEB FiQA2018 (default) + revision: 27a168819829fe9bcd655c2df245fb19452e8e06 + split: test + type: mteb/fiqa + metrics: + - type: main_score + value: 14.184 + - type: map_at_1 + value: 6.694999999999999 + - type: map_at_10 + value: 10.288 + - type: map_at_100 + value: 11.42 + - type: map_at_1000 + value: 11.611 + - type: map_at_20 + value: 10.822 + - type: map_at_3 + value: 8.763 + - type: map_at_5 + value: 9.424000000000001 + - type: mrr_at_1 + value: 12.808641975308642 + - type: mrr_at_10 + value: 17.855428179502255 + - type: mrr_at_100 + value: 18.973068297161007 + - type: mrr_at_1000 + value: 19.0715286472504 + - type: mrr_at_20 + value: 18.453861173947747 + - type: mrr_at_3 + value: 16.023662551440328 + - type: mrr_at_5 + value: 16.78755144032922 + - type: nauc_map_at_1000_diff1 + value: 24.097294182770188 + - type: nauc_map_at_1000_max + value: 3.2685765629171533 + - type: nauc_map_at_1000_std + value: -0.9979553125043357 + - type: nauc_map_at_100_diff1 + value: 24.105146581815614 + - type: nauc_map_at_100_max + value: 3.0585457539422047 + - type: nauc_map_at_100_std + value: -1.1468311670507554 + - type: nauc_map_at_10_diff1 + value: 24.52821887099821 + - type: nauc_map_at_10_max + value: 2.7550316833287845 + - type: nauc_map_at_10_std + value: -2.4843467627121165 + - type: nauc_map_at_1_diff1 + value: 30.95101344161583 + - type: nauc_map_at_1_max + value: -0.5157549348933295 + - type: nauc_map_at_1_std + value: -4.958850144761861 + - type: nauc_map_at_20_diff1 + value: 23.847109306721386 + - type: nauc_map_at_20_max + value: 2.6941597821242254 + - type: nauc_map_at_20_std + value: -2.1229202995449743 + - type: nauc_map_at_3_diff1 + value: 26.478966155221574 + - type: nauc_map_at_3_max + value: 1.0192673396107754 + - type: nauc_map_at_3_std + value: -3.4976902184544514 + - type: nauc_map_at_5_diff1 + value: 25.693752650368516 + - type: nauc_map_at_5_max + value: 3.0475782614572404 + - type: nauc_map_at_5_std + value: -3.0222860925053676 + - type: nauc_mrr_at_1000_diff1 + value: 25.96667963483585 + - type: nauc_mrr_at_1000_max + value: 4.9805028001171365 + - type: nauc_mrr_at_1000_std + value: -5.051980290318263 + - type: nauc_mrr_at_100_diff1 + value: 25.9283221950633 + - type: nauc_mrr_at_100_max + value: 4.913658610096335 + - type: nauc_mrr_at_100_std + value: -5.0647715925530195 + - type: nauc_mrr_at_10_diff1 + value: 26.43424041732111 + - type: nauc_mrr_at_10_max + value: 4.944510711422189 + - type: nauc_mrr_at_10_std + value: -5.508594477775555 + - type: nauc_mrr_at_1_diff1 + value: 32.70658156756328 + - type: nauc_mrr_at_1_max + value: 4.304979000202345 + - type: nauc_mrr_at_1_std + value: -7.012509390453069 + - type: nauc_mrr_at_20_diff1 + value: 25.85251165430012 + - type: nauc_mrr_at_20_max + value: 4.863947427538521 + - type: nauc_mrr_at_20_std + value: -5.4951389153390435 + - type: nauc_mrr_at_3_diff1 + value: 28.17727197125625 + - type: nauc_mrr_at_3_max + value: 3.5466912793013132 + - type: nauc_mrr_at_3_std + value: -5.697472529642202 + - type: nauc_mrr_at_5_diff1 + value: 27.268090979681308 + - type: nauc_mrr_at_5_max + value: 4.255096917087457 + - type: nauc_mrr_at_5_std + value: -6.0629755254741875 + - type: nauc_ndcg_at_1000_diff1 + value: 21.134894210565516 + - type: nauc_ndcg_at_1000_max + value: 8.352509871572998 + - type: nauc_ndcg_at_1000_std + value: 4.917656309055958 + - type: nauc_ndcg_at_100_diff1 + value: 21.22918055007241 + - type: nauc_ndcg_at_100_max + value: 4.962139470825788 + - type: nauc_ndcg_at_100_std + value: 2.6274596028306605 + - type: nauc_ndcg_at_10_diff1 + value: 22.383825724165597 + - type: nauc_ndcg_at_10_max + value: 4.199066108410538 + - type: nauc_ndcg_at_10_std + value: -2.180189858009522 + - type: nauc_ndcg_at_1_diff1 + value: 32.70658156756328 + - type: nauc_ndcg_at_1_max + value: 4.304979000202345 + - type: nauc_ndcg_at_1_std + value: -7.012509390453069 + - type: nauc_ndcg_at_20_diff1 + value: 20.56088961362714 + - type: nauc_ndcg_at_20_max + value: 3.850520965875747 + - type: nauc_ndcg_at_20_std + value: -1.5668477618878296 + - type: nauc_ndcg_at_3_diff1 + value: 25.19652158244801 + - type: nauc_ndcg_at_3_max + value: 2.9159625389970745 + - type: nauc_ndcg_at_3_std + value: -5.262309592366074 + - type: nauc_ndcg_at_5_diff1 + value: 24.387149626781675 + - type: nauc_ndcg_at_5_max + value: 4.582549659190017 + - type: nauc_ndcg_at_5_std + value: -4.2234362048264344 + - type: nauc_precision_at_1000_diff1 + value: 5.818665891519602 + - type: nauc_precision_at_1000_max + value: 21.00907511949278 + - type: nauc_precision_at_1000_std + value: 7.666381615095223 + - type: nauc_precision_at_100_diff1 + value: 12.449744456327732 + - type: nauc_precision_at_100_max + value: 13.892557453999277 + - type: nauc_precision_at_100_std + value: 8.010397294775991 + - type: nauc_precision_at_10_diff1 + value: 16.87222376552928 + - type: nauc_precision_at_10_max + value: 9.553484675352895 + - type: nauc_precision_at_10_std + value: -0.3863038242955302 + - type: nauc_precision_at_1_diff1 + value: 32.70658156756328 + - type: nauc_precision_at_1_max + value: 4.304979000202345 + - type: nauc_precision_at_1_std + value: -7.012509390453069 + - type: nauc_precision_at_20_diff1 + value: 11.634964306167344 + - type: nauc_precision_at_20_max + value: 8.648225058127288 + - type: nauc_precision_at_20_std + value: -0.593009683161987 + - type: nauc_precision_at_3_diff1 + value: 22.40721652841133 + - type: nauc_precision_at_3_max + value: 5.441796570309786 + - type: nauc_precision_at_3_std + value: -5.912794042227691 + - type: nauc_precision_at_5_diff1 + value: 20.994475847749577 + - type: nauc_precision_at_5_max + value: 9.609401441679715 + - type: nauc_precision_at_5_std + value: -5.180422220012539 + - type: nauc_recall_at_1000_diff1 + value: 7.674462981307882 + - type: nauc_recall_at_1000_max + value: 17.216307576877256 + - type: nauc_recall_at_1000_std + value: 29.483396762052937 + - type: nauc_recall_at_100_diff1 + value: 10.932957210915704 + - type: nauc_recall_at_100_max + value: 3.727665900953925 + - type: nauc_recall_at_100_std + value: 12.728089920404603 + - type: nauc_recall_at_10_diff1 + value: 14.59138142453412 + - type: nauc_recall_at_10_max + value: 4.0595316114085085 + - type: nauc_recall_at_10_std + value: 0.31872218135341895 + - type: nauc_recall_at_1_diff1 + value: 30.95101344161583 + - type: nauc_recall_at_1_max + value: -0.5157549348933295 + - type: nauc_recall_at_1_std + value: -4.958850144761861 + - type: nauc_recall_at_20_diff1 + value: 10.581157602615173 + - type: nauc_recall_at_20_max + value: 2.906074522139925 + - type: nauc_recall_at_20_std + value: 1.330036378872078 + - type: nauc_recall_at_3_diff1 + value: 22.289882660691106 + - type: nauc_recall_at_3_max + value: 0.36100797128747864 + - type: nauc_recall_at_3_std + value: -2.6922650407093887 + - type: nauc_recall_at_5_diff1 + value: 19.58957722640653 + - type: nauc_recall_at_5_max + value: 5.462277305621937 + - type: nauc_recall_at_5_std + value: -2.0838117982383966 + - type: ndcg_at_1 + value: 12.809000000000001 + - type: ndcg_at_10 + value: 14.184 + - type: ndcg_at_100 + value: 20.064999999999998 + - type: ndcg_at_1000 + value: 24.413 + - type: ndcg_at_20 + value: 16.03 + - type: ndcg_at_3 + value: 11.829 + - type: ndcg_at_5 + value: 12.348 + - type: precision_at_1 + value: 12.809000000000001 + - type: precision_at_10 + value: 3.997 + - type: precision_at_100 + value: 0.9570000000000001 + - type: precision_at_1000 + value: 0.174 + - type: precision_at_20 + value: 2.685 + - type: precision_at_3 + value: 7.716000000000001 + - type: precision_at_5 + value: 5.679 + - type: recall_at_1 + value: 6.694999999999999 + - type: recall_at_10 + value: 18.394 + - type: recall_at_100 + value: 42.097 + - type: recall_at_1000 + value: 68.631 + - type: recall_at_20 + value: 24.391 + - type: recall_at_3 + value: 10.904 + - type: recall_at_5 + value: 13.141 + task: + type: Retrieval + - dataset: + config: default + name: MTEB HotpotQA (default) + revision: ab518f4d6fcca38d87c25209f94beba119d02014 + split: test + type: mteb/hotpotqa + metrics: + - type: main_score + value: 31.903 + - type: map_at_1 + value: 18.102999999999998 + - type: map_at_10 + value: 24.823 + - type: map_at_100 + value: 25.674999999999997 + - type: map_at_1000 + value: 25.773000000000003 + - type: map_at_20 + value: 25.281 + - type: map_at_3 + value: 22.941 + - type: map_at_5 + value: 23.963 + - type: mrr_at_1 + value: 36.2052667116813 + - type: mrr_at_10 + value: 43.404820852491426 + - type: mrr_at_100 + value: 44.09544455435167 + - type: mrr_at_1000 + value: 44.147517040876984 + - type: mrr_at_20 + value: 43.8108488456634 + - type: mrr_at_3 + value: 41.44722034661278 + - type: mrr_at_5 + value: 42.536349313526934 + - type: nauc_map_at_1000_diff1 + value: 47.27982361373724 + - type: nauc_map_at_1000_max + value: 26.019975780111505 + - type: nauc_map_at_1000_std + value: 9.846281457996731 + - type: nauc_map_at_100_diff1 + value: 47.297153838429566 + - type: nauc_map_at_100_max + value: 26.004987645842707 + - type: nauc_map_at_100_std + value: 9.77464296492678 + - type: nauc_map_at_10_diff1 + value: 47.743548327104726 + - type: nauc_map_at_10_max + value: 25.726168480637163 + - type: nauc_map_at_10_std + value: 8.784313419154504 + - type: nauc_map_at_1_diff1 + value: 58.522274080644486 + - type: nauc_map_at_1_max + value: 26.10698271553083 + - type: nauc_map_at_1_std + value: 3.9695999797031782 + - type: nauc_map_at_20_diff1 + value: 47.51225370171534 + - type: nauc_map_at_20_max + value: 25.88270280062821 + - type: nauc_map_at_20_std + value: 9.29719681497441 + - type: nauc_map_at_3_diff1 + value: 49.916428646760586 + - type: nauc_map_at_3_max + value: 26.148998685574576 + - type: nauc_map_at_3_std + value: 7.5296281829600105 + - type: nauc_map_at_5_diff1 + value: 48.74833155933089 + - type: nauc_map_at_5_max + value: 25.7686251471497 + - type: nauc_map_at_5_std + value: 8.162710914397614 + - type: nauc_mrr_at_1000_diff1 + value: 54.39456285567113 + - type: nauc_mrr_at_1000_max + value: 25.87888554868852 + - type: nauc_mrr_at_1000_std + value: 6.980510424015132 + - type: nauc_mrr_at_100_diff1 + value: 54.383550212913356 + - type: nauc_mrr_at_100_max + value: 25.880465038955343 + - type: nauc_mrr_at_100_std + value: 6.990722947994807 + - type: nauc_mrr_at_10_diff1 + value: 54.40568489034281 + - type: nauc_mrr_at_10_max + value: 25.80484966759375 + - type: nauc_mrr_at_10_std + value: 6.716200895715732 + - type: nauc_mrr_at_1_diff1 + value: 58.522274080644486 + - type: nauc_mrr_at_1_max + value: 26.10698271553083 + - type: nauc_mrr_at_1_std + value: 3.9695999797031782 + - type: nauc_mrr_at_20_diff1 + value: 54.38354933580429 + - type: nauc_mrr_at_20_max + value: 25.82697081406741 + - type: nauc_mrr_at_20_std + value: 6.840324995589092 + - type: nauc_mrr_at_3_diff1 + value: 55.20845972552233 + - type: nauc_mrr_at_3_max + value: 26.133818717512252 + - type: nauc_mrr_at_3_std + value: 6.18610273343223 + - type: nauc_mrr_at_5_diff1 + value: 54.77569448991443 + - type: nauc_mrr_at_5_max + value: 25.909722538158714 + - type: nauc_mrr_at_5_std + value: 6.457350156713465 + - type: nauc_ndcg_at_1000_diff1 + value: 45.31342042774975 + - type: nauc_ndcg_at_1000_max + value: 26.460991437144067 + - type: nauc_ndcg_at_1000_std + value: 14.494000388772085 + - type: nauc_ndcg_at_100_diff1 + value: 45.734953321121324 + - type: nauc_ndcg_at_100_max + value: 26.347893077226963 + - type: nauc_ndcg_at_100_std + value: 13.486443287557954 + - type: nauc_ndcg_at_10_diff1 + value: 47.2712801149981 + - type: nauc_ndcg_at_10_max + value: 25.362835595556977 + - type: nauc_ndcg_at_10_std + value: 9.703109157709564 + - type: nauc_ndcg_at_1_diff1 + value: 58.522274080644486 + - type: nauc_ndcg_at_1_max + value: 26.10698271553083 + - type: nauc_ndcg_at_1_std + value: 3.9695999797031782 + - type: nauc_ndcg_at_20_diff1 + value: 46.68362518042518 + - type: nauc_ndcg_at_20_max + value: 25.679514271642507 + - type: nauc_ndcg_at_20_std + value: 10.896046904643255 + - type: nauc_ndcg_at_3_diff1 + value: 50.58611283409585 + - type: nauc_ndcg_at_3_max + value: 26.118344876777638 + - type: nauc_ndcg_at_3_std + value: 7.795982972448585 + - type: nauc_ndcg_at_5_diff1 + value: 49.03878464866394 + - type: nauc_ndcg_at_5_max + value: 25.558039532800663 + - type: nauc_ndcg_at_5_std + value: 8.603068871106622 + - type: nauc_precision_at_1000_diff1 + value: 14.101040732969647 + - type: nauc_precision_at_1000_max + value: 20.86409428068311 + - type: nauc_precision_at_1000_std + value: 33.83928771739056 + - type: nauc_precision_at_100_diff1 + value: 23.60806046646006 + - type: nauc_precision_at_100_max + value: 22.79001814313364 + - type: nauc_precision_at_100_std + value: 26.623524166212675 + - type: nauc_precision_at_10_diff1 + value: 34.929881985998954 + - type: nauc_precision_at_10_max + value: 22.30282244931993 + - type: nauc_precision_at_10_std + value: 13.647922319906481 + - type: nauc_precision_at_1_diff1 + value: 58.522274080644486 + - type: nauc_precision_at_1_max + value: 26.10698271553083 + - type: nauc_precision_at_1_std + value: 3.9695999797031782 + - type: nauc_precision_at_20_diff1 + value: 32.184958004379226 + - type: nauc_precision_at_20_max + value: 22.771289682864097 + - type: nauc_precision_at_20_std + value: 17.122428431864808 + - type: nauc_precision_at_3_diff1 + value: 45.638742470642235 + - type: nauc_precision_at_3_max + value: 25.791252930433156 + - type: nauc_precision_at_3_std + value: 9.990734907218188 + - type: nauc_precision_at_5_diff1 + value: 41.42511675226 + - type: nauc_precision_at_5_max + value: 24.014233009915195 + - type: nauc_precision_at_5_std + value: 11.40080345708828 + - type: nauc_recall_at_1000_diff1 + value: 14.101040732969633 + - type: nauc_recall_at_1000_max + value: 20.864094280683226 + - type: nauc_recall_at_1000_std + value: 33.839287717390626 + - type: nauc_recall_at_100_diff1 + value: 23.608060466459975 + - type: nauc_recall_at_100_max + value: 22.79001814313358 + - type: nauc_recall_at_100_std + value: 26.623524166212597 + - type: nauc_recall_at_10_diff1 + value: 34.929881985998975 + - type: nauc_recall_at_10_max + value: 22.302822449319958 + - type: nauc_recall_at_10_std + value: 13.647922319906506 + - type: nauc_recall_at_1_diff1 + value: 58.522274080644486 + - type: nauc_recall_at_1_max + value: 26.10698271553083 + - type: nauc_recall_at_1_std + value: 3.9695999797031782 + - type: nauc_recall_at_20_diff1 + value: 32.18495800437926 + - type: nauc_recall_at_20_max + value: 22.771289682864094 + - type: nauc_recall_at_20_std + value: 17.12242843186481 + - type: nauc_recall_at_3_diff1 + value: 45.63874247064222 + - type: nauc_recall_at_3_max + value: 25.791252930433174 + - type: nauc_recall_at_3_std + value: 9.990734907218165 + - type: nauc_recall_at_5_diff1 + value: 41.42511675226002 + - type: nauc_recall_at_5_max + value: 24.014233009915227 + - type: nauc_recall_at_5_std + value: 11.400803457088296 + - type: ndcg_at_1 + value: 36.205 + - type: ndcg_at_10 + value: 31.903 + - type: ndcg_at_100 + value: 35.859 + - type: ndcg_at_1000 + value: 38.218999999999994 + - type: ndcg_at_20 + value: 33.379999999999995 + - type: ndcg_at_3 + value: 28.249000000000002 + - type: ndcg_at_5 + value: 29.981 + - type: precision_at_1 + value: 36.205 + - type: precision_at_10 + value: 6.959 + - type: precision_at_100 + value: 1.013 + - type: precision_at_1000 + value: 0.133 + - type: precision_at_20 + value: 3.955 + - type: precision_at_3 + value: 17.677 + - type: precision_at_5 + value: 11.978 + - type: recall_at_1 + value: 18.102999999999998 + - type: recall_at_10 + value: 34.794000000000004 + - type: recall_at_100 + value: 50.62799999999999 + - type: recall_at_1000 + value: 66.39399999999999 + - type: recall_at_20 + value: 39.554 + - type: recall_at_3 + value: 26.516000000000002 + - type: recall_at_5 + value: 29.946 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ImdbClassification (default) + revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 + split: test + type: mteb/imdb + metrics: + - type: accuracy + value: 70.78880000000001 + - type: ap + value: 65.03883652194777 + - type: ap_weighted + value: 65.03883652194777 + - type: f1 + value: 70.58556203674722 + - type: f1_weighted + value: 70.58556203674722 + - type: main_score + value: 70.78880000000001 + task: + type: Classification + - dataset: + config: default + name: MTEB MSMARCO (default) + revision: c5a29a104738b98a9e76336939199e264163d4a0 + split: test + type: mteb/msmarco + metrics: + - type: main_score + value: 32.823 + - type: map_at_1 + value: 0.9860000000000001 + - type: map_at_10 + value: 5.976 + - type: map_at_100 + value: 15.404000000000002 + - type: map_at_1000 + value: 19.567999999999998 + - type: map_at_20 + value: 9.109 + - type: map_at_3 + value: 2.418 + - type: map_at_5 + value: 3.864 + - type: mrr_at_1 + value: 48.837209302325576 + - type: mrr_at_10 + value: 62.2093023255814 + - type: mrr_at_100 + value: 62.27770177838576 + - type: mrr_at_1000 + value: 62.28904607787528 + - type: mrr_at_20 + value: 62.2093023255814 + - type: mrr_at_3 + value: 58.914728682170534 + - type: mrr_at_5 + value: 61.240310077519375 + - type: nauc_map_at_1000_diff1 + value: 8.751729457031873 + - type: nauc_map_at_1000_max + value: 60.04896551891142 + - type: nauc_map_at_1000_std + value: 53.077212935277466 + - type: nauc_map_at_100_diff1 + value: 6.922687490370606 + - type: nauc_map_at_100_max + value: 49.65059065309455 + - type: nauc_map_at_100_std + value: 42.88292911032164 + - type: nauc_map_at_10_diff1 + value: -0.3783247628730733 + - type: nauc_map_at_10_max + value: 26.69355085391205 + - type: nauc_map_at_10_std + value: 21.035834730108895 + - type: nauc_map_at_1_diff1 + value: -12.382912648953685 + - type: nauc_map_at_1_max + value: 15.103511392499733 + - type: nauc_map_at_1_std + value: 10.646234291111401 + - type: nauc_map_at_20_diff1 + value: 2.470781745822043 + - type: nauc_map_at_20_max + value: 30.876111163664127 + - type: nauc_map_at_20_std + value: 24.9809791173482 + - type: nauc_map_at_3_diff1 + value: -8.321628149092982 + - type: nauc_map_at_3_max + value: 19.4280666057206 + - type: nauc_map_at_3_std + value: 17.149294671189587 + - type: nauc_map_at_5_diff1 + value: -6.013325870911972 + - type: nauc_map_at_5_max + value: 21.96291629437915 + - type: nauc_map_at_5_std + value: 18.287688144122864 + - type: nauc_mrr_at_1000_diff1 + value: -6.55908560475509 + - type: nauc_mrr_at_1000_max + value: 63.7357804040643 + - type: nauc_mrr_at_1000_std + value: 45.246136682418225 + - type: nauc_mrr_at_100_diff1 + value: -6.5274276607751185 + - type: nauc_mrr_at_100_max + value: 63.7503664464894 + - type: nauc_mrr_at_100_std + value: 45.25017528433923 + - type: nauc_mrr_at_10_diff1 + value: -6.379884653902308 + - type: nauc_mrr_at_10_max + value: 63.79070493015706 + - type: nauc_mrr_at_10_std + value: 45.365484352508275 + - type: nauc_mrr_at_1_diff1 + value: -17.84019927997559 + - type: nauc_mrr_at_1_max + value: 58.52755657150483 + - type: nauc_mrr_at_1_std + value: 41.73471680745748 + - type: nauc_mrr_at_20_diff1 + value: -6.379884653902308 + - type: nauc_mrr_at_20_max + value: 63.79070493015706 + - type: nauc_mrr_at_20_std + value: 45.365484352508275 + - type: nauc_mrr_at_3_diff1 + value: -3.9083100270657716 + - type: nauc_mrr_at_3_max + value: 62.18741183069903 + - type: nauc_mrr_at_3_std + value: 43.295658470021436 + - type: nauc_mrr_at_5_diff1 + value: -7.1215738827525925 + - type: nauc_mrr_at_5_max + value: 62.60576741721094 + - type: nauc_mrr_at_5_std + value: 41.880403498948716 + - type: nauc_ndcg_at_1000_diff1 + value: 7.332595234385046 + - type: nauc_ndcg_at_1000_max + value: 64.32975587377672 + - type: nauc_ndcg_at_1000_std + value: 61.51743128459955 + - type: nauc_ndcg_at_100_diff1 + value: 1.4494382437319793 + - type: nauc_ndcg_at_100_max + value: 57.874195533402286 + - type: nauc_ndcg_at_100_std + value: 50.888248375236614 + - type: nauc_ndcg_at_10_diff1 + value: -0.7315495724138182 + - type: nauc_ndcg_at_10_max + value: 54.96996776097349 + - type: nauc_ndcg_at_10_std + value: 45.27121651582023 + - type: nauc_ndcg_at_1_diff1 + value: -15.056696340171236 + - type: nauc_ndcg_at_1_max + value: 47.61677629151873 + - type: nauc_ndcg_at_1_std + value: 37.02896277566421 + - type: nauc_ndcg_at_20_diff1 + value: 0.8233405712285105 + - type: nauc_ndcg_at_20_max + value: 58.96827303834117 + - type: nauc_ndcg_at_20_std + value: 50.86458633201503 + - type: nauc_ndcg_at_3_diff1 + value: -4.276906949610996 + - type: nauc_ndcg_at_3_max + value: 48.31308632024211 + - type: nauc_ndcg_at_3_std + value: 38.455388100400725 + - type: nauc_ndcg_at_5_diff1 + value: -1.3421422620129042 + - type: nauc_ndcg_at_5_max + value: 51.9123655054488 + - type: nauc_ndcg_at_5_std + value: 41.674609662059375 + - type: nauc_precision_at_1000_diff1 + value: 6.372456647149425 + - type: nauc_precision_at_1000_max + value: 56.78532698940627 + - type: nauc_precision_at_1000_std + value: 52.46483738053407 + - type: nauc_precision_at_100_diff1 + value: 6.577021351469274 + - type: nauc_precision_at_100_max + value: 63.76823129393031 + - type: nauc_precision_at_100_std + value: 56.34082227414525 + - type: nauc_precision_at_10_diff1 + value: 12.569719755250647 + - type: nauc_precision_at_10_max + value: 64.12253935059921 + - type: nauc_precision_at_10_std + value: 53.28375185847503 + - type: nauc_precision_at_1_diff1 + value: -17.84019927997559 + - type: nauc_precision_at_1_max + value: 58.52755657150483 + - type: nauc_precision_at_1_std + value: 41.73471680745748 + - type: nauc_precision_at_20_diff1 + value: 12.782123104838256 + - type: nauc_precision_at_20_max + value: 64.62717668527766 + - type: nauc_precision_at_20_std + value: 56.713030305356725 + - type: nauc_precision_at_3_diff1 + value: 1.0137979188426132 + - type: nauc_precision_at_3_max + value: 59.00556447771772 + - type: nauc_precision_at_3_std + value: 48.02337913753564 + - type: nauc_precision_at_5_diff1 + value: 6.616675723116447 + - type: nauc_precision_at_5_max + value: 62.87219840100712 + - type: nauc_precision_at_5_std + value: 49.86527880916804 + - type: nauc_recall_at_1000_diff1 + value: 9.556219515257334 + - type: nauc_recall_at_1000_max + value: 57.24110273060209 + - type: nauc_recall_at_1000_std + value: 57.56303957604837 + - type: nauc_recall_at_100_diff1 + value: 3.5160058971838013 + - type: nauc_recall_at_100_max + value: 44.6508691428123 + - type: nauc_recall_at_100_std + value: 40.469866482826845 + - type: nauc_recall_at_10_diff1 + value: 2.139024250495015 + - type: nauc_recall_at_10_max + value: 20.006408713083758 + - type: nauc_recall_at_10_std + value: 15.249248068351031 + - type: nauc_recall_at_1_diff1 + value: -12.382912648953685 + - type: nauc_recall_at_1_max + value: 15.103511392499733 + - type: nauc_recall_at_1_std + value: 10.646234291111401 + - type: nauc_recall_at_20_diff1 + value: 2.765059666028026 + - type: nauc_recall_at_20_max + value: 25.807810834205707 + - type: nauc_recall_at_20_std + value: 21.3906188095389 + - type: nauc_recall_at_3_diff1 + value: -2.901694619296191 + - type: nauc_recall_at_3_max + value: 19.674783823606212 + - type: nauc_recall_at_3_std + value: 17.13722598183868 + - type: nauc_recall_at_5_diff1 + value: -4.0200493920102875 + - type: nauc_recall_at_5_max + value: 17.514540336925517 + - type: nauc_recall_at_5_std + value: 12.433202741785959 + - type: ndcg_at_1 + value: 31.395 + - type: ndcg_at_10 + value: 32.823 + - type: ndcg_at_100 + value: 32.096000000000004 + - type: ndcg_at_1000 + value: 40.994 + - type: ndcg_at_20 + value: 31.691000000000003 + - type: ndcg_at_3 + value: 32.53 + - type: ndcg_at_5 + value: 34.136 + - type: precision_at_1 + value: 48.837 + - type: precision_at_10 + value: 44.186 + - type: precision_at_100 + value: 21.302 + - type: precision_at_1000 + value: 4.607 + - type: precision_at_20 + value: 38.372 + - type: precision_at_3 + value: 50.388 + - type: precision_at_5 + value: 50.698 + - type: recall_at_1 + value: 0.9860000000000001 + - type: recall_at_10 + value: 7.585 + - type: recall_at_100 + value: 27.534999999999997 + - type: recall_at_1000 + value: 51.769 + - type: recall_at_20 + value: 12.361 + - type: recall_at_3 + value: 2.625 + - type: recall_at_5 + value: 4.593 + task: + type: Retrieval + - dataset: + config: en + name: MTEB MTOPDomainClassification (en) + revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf + split: test + type: mteb/mtop_domain + metrics: + - type: accuracy + value: 85.6155950752394 + - type: f1 + value: 84.61299351825656 + - type: f1_weighted + value: 85.62566329953071 + - type: main_score + value: 85.6155950752394 + task: + type: Classification + - dataset: + config: en + name: MTEB MTOPIntentClassification (en) + revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba + split: test + type: mteb/mtop_intent + metrics: + - type: accuracy + value: 51.507067943456455 + - type: f1 + value: 33.73329265389269 + - type: f1_weighted + value: 54.70051724359958 + - type: main_score + value: 51.507067943456455 + task: + type: Classification + - dataset: + config: en + name: MTEB MassiveIntentClassification (en) + revision: 4672e20407010da34463acc759c162ca9734bca6 + split: test + type: mteb/amazon_massive_intent + metrics: + - type: accuracy + value: 60.67249495628783 + - type: f1 + value: 58.55710441567331 + - type: f1_weighted + value: 59.4792418446047 + - type: main_score + value: 60.67249495628783 + task: + type: Classification + - dataset: + config: en + name: MTEB MassiveScenarioClassification (en) + revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 + split: test + type: mteb/amazon_massive_scenario + metrics: + - type: accuracy + value: 66.94687289845326 + - type: f1 + value: 65.55262619224948 + - type: f1_weighted + value: 66.7594041012024 + - type: main_score + value: 66.94687289845326 + task: + type: Classification + - dataset: + config: default + name: MTEB MedrxivClusteringP2P (default) + revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 + split: test + type: mteb/medrxiv-clustering-p2p + metrics: + - type: main_score + value: 27.033544101894037 + - type: v_measure + value: 27.033544101894037 + - type: v_measure_std + value: 1.3875075110570252 + task: + type: Clustering + - dataset: + config: default + name: MTEB MedrxivClusteringS2S (default) + revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 + split: test + type: mteb/medrxiv-clustering-s2s + metrics: + - type: main_score + value: 23.235475122772748 + - type: v_measure + value: 23.235475122772748 + - type: v_measure_std + value: 1.3669816155807724 + task: + type: Clustering + - dataset: + config: default + name: MTEB MindSmallReranking (default) + revision: 59042f120c80e8afa9cdbb224f67076cec0fc9a7 + split: test + type: mteb/mind_small + metrics: + - type: main_score + value: 29.025211119858856 + - type: map + value: 29.025211119858856 + - type: mrr + value: 29.72218362379489 + - type: nAUC_map_diff1 + value: 10.344777333382883 + - type: nAUC_map_max + value: -23.666261656057035 + - type: nAUC_map_std + value: -8.98726774731994 + - type: nAUC_mrr_diff1 + value: 10.267778513424473 + - type: nAUC_mrr_max + value: -18.006089017770165 + - type: nAUC_mrr_std + value: -6.484973234862847 + task: + type: Reranking + - dataset: + config: default + name: MTEB NFCorpus (default) + revision: ec0fa4fe99da2ff19ca1214b7966684033a58814 + split: test + type: mteb/nfcorpus + metrics: + - type: main_score + value: 22.613 + - type: map_at_1 + value: 3.4189999999999996 + - type: map_at_10 + value: 6.812 + - type: map_at_100 + value: 8.767999999999999 + - type: map_at_1000 + value: 9.82 + - type: map_at_20 + value: 7.593999999999999 + - type: map_at_3 + value: 5.281000000000001 + - type: map_at_5 + value: 5.992999999999999 + - type: mrr_at_1 + value: 32.81733746130031 + - type: mrr_at_10 + value: 41.73694039019116 + - type: mrr_at_100 + value: 42.44672811430311 + - type: mrr_at_1000 + value: 42.51492426020226 + - type: mrr_at_20 + value: 42.15273887004662 + - type: mrr_at_3 + value: 39.42208462332301 + - type: mrr_at_5 + value: 40.90815273477812 + - type: nauc_map_at_1000_diff1 + value: 32.51914859153408 + - type: nauc_map_at_1000_max + value: 28.50731007634339 + - type: nauc_map_at_1000_std + value: 21.122290838876197 + - type: nauc_map_at_100_diff1 + value: 34.0356163812035 + - type: nauc_map_at_100_max + value: 26.984927084496746 + - type: nauc_map_at_100_std + value: 17.16714022131354 + - type: nauc_map_at_10_diff1 + value: 38.049956057842856 + - type: nauc_map_at_10_max + value: 21.689889302108227 + - type: nauc_map_at_10_std + value: 8.900716391912274 + - type: nauc_map_at_1_diff1 + value: 43.69318427182068 + - type: nauc_map_at_1_max + value: 9.993398066053729 + - type: nauc_map_at_1_std + value: 1.6998173556132434 + - type: nauc_map_at_20_diff1 + value: 35.65581914391132 + - type: nauc_map_at_20_max + value: 24.418344690964613 + - type: nauc_map_at_20_std + value: 12.521959576048808 + - type: nauc_map_at_3_diff1 + value: 43.16156145914653 + - type: nauc_map_at_3_max + value: 17.094747748444377 + - type: nauc_map_at_3_std + value: 4.067978802659887 + - type: nauc_map_at_5_diff1 + value: 41.407965249135806 + - type: nauc_map_at_5_max + value: 19.90265827157068 + - type: nauc_map_at_5_std + value: 6.339864622044265 + - type: nauc_mrr_at_1000_diff1 + value: 26.099863745051945 + - type: nauc_mrr_at_1000_max + value: 30.451300657305747 + - type: nauc_mrr_at_1000_std + value: 20.99147829162304 + - type: nauc_mrr_at_100_diff1 + value: 26.09622707115036 + - type: nauc_mrr_at_100_max + value: 30.47822047389741 + - type: nauc_mrr_at_100_std + value: 21.020356459106182 + - type: nauc_mrr_at_10_diff1 + value: 25.767563178245727 + - type: nauc_mrr_at_10_max + value: 30.317099022190092 + - type: nauc_mrr_at_10_std + value: 21.039453634175945 + - type: nauc_mrr_at_1_diff1 + value: 27.422277143717373 + - type: nauc_mrr_at_1_max + value: 23.638411521485935 + - type: nauc_mrr_at_1_std + value: 15.499293879835903 + - type: nauc_mrr_at_20_diff1 + value: 25.98915185113393 + - type: nauc_mrr_at_20_max + value: 30.51502999286555 + - type: nauc_mrr_at_20_std + value: 21.007107746436386 + - type: nauc_mrr_at_3_diff1 + value: 26.324556538009535 + - type: nauc_mrr_at_3_max + value: 29.369067665865117 + - type: nauc_mrr_at_3_std + value: 20.588832297661824 + - type: nauc_mrr_at_5_diff1 + value: 26.460468401069974 + - type: nauc_mrr_at_5_max + value: 29.86644460025973 + - type: nauc_mrr_at_5_std + value: 19.849343846164693 + - type: nauc_ndcg_at_1000_diff1 + value: 26.137174251660817 + - type: nauc_ndcg_at_1000_max + value: 36.88679053188914 + - type: nauc_ndcg_at_1000_std + value: 28.545299901436966 + - type: nauc_ndcg_at_100_diff1 + value: 24.38235346166006 + - type: nauc_ndcg_at_100_max + value: 32.35934908511791 + - type: nauc_ndcg_at_100_std + value: 26.621500665622555 + - type: nauc_ndcg_at_10_diff1 + value: 21.346110864966878 + - type: nauc_ndcg_at_10_max + value: 29.201870789543488 + - type: nauc_ndcg_at_10_std + value: 27.996224821771143 + - type: nauc_ndcg_at_1_diff1 + value: 27.82064172474196 + - type: nauc_ndcg_at_1_max + value: 22.320782468431528 + - type: nauc_ndcg_at_1_std + value: 17.01269970506558 + - type: nauc_ndcg_at_20_diff1 + value: 22.695086276964542 + - type: nauc_ndcg_at_20_max + value: 30.14781423581441 + - type: nauc_ndcg_at_20_std + value: 29.750857717833657 + - type: nauc_ndcg_at_3_diff1 + value: 23.23035291802125 + - type: nauc_ndcg_at_3_max + value: 27.951166119373728 + - type: nauc_ndcg_at_3_std + value: 20.969642025523797 + - type: nauc_ndcg_at_5_diff1 + value: 22.275733427660523 + - type: nauc_ndcg_at_5_max + value: 29.21633819667062 + - type: nauc_ndcg_at_5_std + value: 23.258878040008515 + - type: nauc_precision_at_1000_diff1 + value: -4.522644786677741 + - type: nauc_precision_at_1000_max + value: 17.54188976143716 + - type: nauc_precision_at_1000_std + value: 43.566922007655045 + - type: nauc_precision_at_100_diff1 + value: 0.7117884922496146 + - type: nauc_precision_at_100_max + value: 27.073214361431496 + - type: nauc_precision_at_100_std + value: 46.737829736269134 + - type: nauc_precision_at_10_diff1 + value: 8.359859618942956 + - type: nauc_precision_at_10_max + value: 31.75752536248692 + - type: nauc_precision_at_10_std + value: 36.89066103106389 + - type: nauc_precision_at_1_diff1 + value: 27.422277143717373 + - type: nauc_precision_at_1_max + value: 23.638411521485935 + - type: nauc_precision_at_1_std + value: 15.499293879835903 + - type: nauc_precision_at_20_diff1 + value: 5.055347818306108 + - type: nauc_precision_at_20_max + value: 32.721802908008186 + - type: nauc_precision_at_20_std + value: 42.89763536496222 + - type: nauc_precision_at_3_diff1 + value: 17.848359825929847 + - type: nauc_precision_at_3_max + value: 31.47671680933972 + - type: nauc_precision_at_3_std + value: 23.67581982693161 + - type: nauc_precision_at_5_diff1 + value: 13.851675563275661 + - type: nauc_precision_at_5_max + value: 33.661786215343746 + - type: nauc_precision_at_5_std + value: 28.482203360698065 + - type: nauc_recall_at_1000_diff1 + value: 13.961711120759071 + - type: nauc_recall_at_1000_max + value: 19.382624712902455 + - type: nauc_recall_at_1000_std + value: 14.558738251676376 + - type: nauc_recall_at_100_diff1 + value: 18.32951532159328 + - type: nauc_recall_at_100_max + value: 23.974271967720846 + - type: nauc_recall_at_100_std + value: 14.252037430687484 + - type: nauc_recall_at_10_diff1 + value: 28.95671818041367 + - type: nauc_recall_at_10_max + value: 19.625268127300917 + - type: nauc_recall_at_10_std + value: 9.165959351962458 + - type: nauc_recall_at_1_diff1 + value: 43.69318427182068 + - type: nauc_recall_at_1_max + value: 9.993398066053729 + - type: nauc_recall_at_1_std + value: 1.6998173556132434 + - type: nauc_recall_at_20_diff1 + value: 24.956992636170295 + - type: nauc_recall_at_20_max + value: 19.47601942861755 + - type: nauc_recall_at_20_std + value: 10.507076581980725 + - type: nauc_recall_at_3_diff1 + value: 40.91167574965666 + - type: nauc_recall_at_3_max + value: 19.0859559383436 + - type: nauc_recall_at_3_std + value: 6.051536150937656 + - type: nauc_recall_at_5_diff1 + value: 36.54933600909664 + - type: nauc_recall_at_5_max + value: 20.232938987005646 + - type: nauc_recall_at_5_std + value: 7.413199389324412 + - type: ndcg_at_1 + value: 30.805 + - type: ndcg_at_10 + value: 22.613 + - type: ndcg_at_100 + value: 20.928 + - type: ndcg_at_1000 + value: 29.695 + - type: ndcg_at_20 + value: 21.09 + - type: ndcg_at_3 + value: 26.377 + - type: ndcg_at_5 + value: 24.595 + - type: precision_at_1 + value: 32.817 + - type: precision_at_10 + value: 16.966 + - type: precision_at_100 + value: 5.811 + - type: precision_at_1000 + value: 1.806 + - type: precision_at_20 + value: 12.802 + - type: precision_at_3 + value: 24.871 + - type: precision_at_5 + value: 21.3 + - type: recall_at_1 + value: 3.4189999999999996 + - type: recall_at_10 + value: 9.883000000000001 + - type: recall_at_100 + value: 22.392 + - type: recall_at_1000 + value: 54.018 + - type: recall_at_20 + value: 13.106000000000002 + - type: recall_at_3 + value: 6.203 + - type: recall_at_5 + value: 7.688000000000001 + task: + type: Retrieval + - dataset: + config: default + name: MTEB NQ (default) + revision: b774495ed302d8c44a3a7ea25c90dbce03968f31 + split: test + type: mteb/nq + metrics: + - type: main_score + value: 17.163999999999998 + - type: map_at_1 + value: 7.292999999999999 + - type: map_at_10 + value: 13.206000000000001 + - type: map_at_100 + value: 14.368 + - type: map_at_1000 + value: 14.468 + - type: map_at_20 + value: 13.825000000000001 + - type: map_at_3 + value: 10.949 + - type: map_at_5 + value: 12.110999999999999 + - type: mrr_at_1 + value: 8.487833140208574 + - type: mrr_at_10 + value: 14.675159557836242 + - type: mrr_at_100 + value: 15.779595948147401 + - type: mrr_at_1000 + value: 15.86576620262906 + - type: mrr_at_20 + value: 15.296918959194382 + - type: mrr_at_3 + value: 12.369640787948995 + - type: mrr_at_5 + value: 13.58198146002313 + - type: nauc_map_at_1000_diff1 + value: 19.323868662545987 + - type: nauc_map_at_1000_max + value: 13.767084099660302 + - type: nauc_map_at_1000_std + value: 5.232483665156492 + - type: nauc_map_at_100_diff1 + value: 19.345070681167932 + - type: nauc_map_at_100_max + value: 13.742816206910161 + - type: nauc_map_at_100_std + value: 5.153409411301769 + - type: nauc_map_at_10_diff1 + value: 19.659707100205903 + - type: nauc_map_at_10_max + value: 13.131492785425097 + - type: nauc_map_at_10_std + value: 3.7953216544067767 + - type: nauc_map_at_1_diff1 + value: 23.621537828194803 + - type: nauc_map_at_1_max + value: 9.359321805095224 + - type: nauc_map_at_1_std + value: 0.9564844869375119 + - type: nauc_map_at_20_diff1 + value: 19.41548662256754 + - type: nauc_map_at_20_max + value: 13.44112402133028 + - type: nauc_map_at_20_std + value: 4.526186657561462 + - type: nauc_map_at_3_diff1 + value: 20.874144569488028 + - type: nauc_map_at_3_max + value: 10.59497020339296 + - type: nauc_map_at_3_std + value: 2.127061943857287 + - type: nauc_map_at_5_diff1 + value: 20.212151988624548 + - type: nauc_map_at_5_max + value: 11.889866875966785 + - type: nauc_map_at_5_std + value: 2.7199819521456154 + - type: nauc_mrr_at_1000_diff1 + value: 18.421603253063438 + - type: nauc_mrr_at_1000_max + value: 13.192373130005967 + - type: nauc_mrr_at_1000_std + value: 6.582095129043891 + - type: nauc_mrr_at_100_diff1 + value: 18.445175537134432 + - type: nauc_mrr_at_100_max + value: 13.177218814359042 + - type: nauc_mrr_at_100_std + value: 6.540820393254043 + - type: nauc_mrr_at_10_diff1 + value: 18.67543777577155 + - type: nauc_mrr_at_10_max + value: 12.698796909888438 + - type: nauc_mrr_at_10_std + value: 5.548542219175191 + - type: nauc_mrr_at_1_diff1 + value: 22.405046326183907 + - type: nauc_mrr_at_1_max + value: 9.290311837698303 + - type: nauc_mrr_at_1_std + value: 3.772169020187191 + - type: nauc_mrr_at_20_diff1 + value: 18.406790295652588 + - type: nauc_mrr_at_20_max + value: 12.982208065169374 + - type: nauc_mrr_at_20_std + value: 6.084379018636953 + - type: nauc_mrr_at_3_diff1 + value: 19.761212114449446 + - type: nauc_mrr_at_3_max + value: 10.586228375739465 + - type: nauc_mrr_at_3_std + value: 4.454489369742695 + - type: nauc_mrr_at_5_diff1 + value: 19.118475150105894 + - type: nauc_mrr_at_5_max + value: 11.58082985340595 + - type: nauc_mrr_at_5_std + value: 4.766402493324253 + - type: nauc_ndcg_at_1000_diff1 + value: 16.96375429586292 + - type: nauc_ndcg_at_1000_max + value: 18.826013150783883 + - type: nauc_ndcg_at_1000_std + value: 12.9887940091125 + - type: nauc_ndcg_at_100_diff1 + value: 17.377295880274822 + - type: nauc_ndcg_at_100_max + value: 18.099418324015964 + - type: nauc_ndcg_at_100_std + value: 11.394891190564095 + - type: nauc_ndcg_at_10_diff1 + value: 18.06378127446637 + - type: nauc_ndcg_at_10_max + value: 15.297698965310232 + - type: nauc_ndcg_at_10_std + value: 5.717861719899932 + - type: nauc_ndcg_at_1_diff1 + value: 22.405046326183907 + - type: nauc_ndcg_at_1_max + value: 9.290311837698303 + - type: nauc_ndcg_at_1_std + value: 3.772169020187191 + - type: nauc_ndcg_at_20_diff1 + value: 17.34734627220401 + - type: nauc_ndcg_at_20_max + value: 16.20778833559378 + - type: nauc_ndcg_at_20_std + value: 7.749314013181468 + - type: nauc_ndcg_at_3_diff1 + value: 20.151547280803808 + - type: nauc_ndcg_at_3_max + value: 10.886180458060574 + - type: nauc_ndcg_at_3_std + value: 2.855909481970702 + - type: nauc_ndcg_at_5_diff1 + value: 19.08860016992726 + - type: nauc_ndcg_at_5_max + value: 12.76124542662454 + - type: nauc_ndcg_at_5_std + value: 3.6782880617792566 + - type: nauc_precision_at_1000_diff1 + value: 3.4126117328409635 + - type: nauc_precision_at_1000_max + value: 26.144517857081162 + - type: nauc_precision_at_1000_std + value: 35.15957366598821 + - type: nauc_precision_at_100_diff1 + value: 10.61994042439232 + - type: nauc_precision_at_100_max + value: 25.44638747527479 + - type: nauc_precision_at_100_std + value: 27.20279863003429 + - type: nauc_precision_at_10_diff1 + value: 15.051182307262142 + - type: nauc_precision_at_10_max + value: 19.818107620262378 + - type: nauc_precision_at_10_std + value: 10.84691010517258 + - type: nauc_precision_at_1_diff1 + value: 22.405046326183907 + - type: nauc_precision_at_1_max + value: 9.290311837698303 + - type: nauc_precision_at_1_std + value: 3.772169020187191 + - type: nauc_precision_at_20_diff1 + value: 12.638984559185824 + - type: nauc_precision_at_20_max + value: 21.391773469704304 + - type: nauc_precision_at_20_std + value: 16.018277911853563 + - type: nauc_precision_at_3_diff1 + value: 18.573099204831646 + - type: nauc_precision_at_3_max + value: 11.81684575514176 + - type: nauc_precision_at_3_std + value: 5.046334944705079 + - type: nauc_precision_at_5_diff1 + value: 17.149349734404016 + - type: nauc_precision_at_5_max + value: 15.197844652143155 + - type: nauc_precision_at_5_std + value: 6.453417880704823 + - type: nauc_recall_at_1000_diff1 + value: 9.836204388910634 + - type: nauc_recall_at_1000_max + value: 40.64943837622281 + - type: nauc_recall_at_1000_std + value: 46.01484423115387 + - type: nauc_recall_at_100_diff1 + value: 13.656477700316763 + - type: nauc_recall_at_100_max + value: 28.768748963828845 + - type: nauc_recall_at_100_std + value: 26.247014408258778 + - type: nauc_recall_at_10_diff1 + value: 15.055017670873397 + - type: nauc_recall_at_10_max + value: 19.424431660534115 + - type: nauc_recall_at_10_std + value: 8.12141284246005 + - type: nauc_recall_at_1_diff1 + value: 23.621537828194803 + - type: nauc_recall_at_1_max + value: 9.359321805095224 + - type: nauc_recall_at_1_std + value: 0.9564844869375119 + - type: nauc_recall_at_20_diff1 + value: 13.412421761546462 + - type: nauc_recall_at_20_max + value: 21.276341065435446 + - type: nauc_recall_at_20_std + value: 12.847302508235089 + - type: nauc_recall_at_3_diff1 + value: 18.915998121406723 + - type: nauc_recall_at_3_max + value: 11.235299022137816 + - type: nauc_recall_at_3_std + value: 2.6771446726975703 + - type: nauc_recall_at_5_diff1 + value: 16.998490481118615 + - type: nauc_recall_at_5_max + value: 14.326029592435003 + - type: nauc_recall_at_5_std + value: 4.083735570817751 + - type: ndcg_at_1 + value: 8.488 + - type: ndcg_at_10 + value: 17.163999999999998 + - type: ndcg_at_100 + value: 23.166 + - type: ndcg_at_1000 + value: 25.899 + - type: ndcg_at_20 + value: 19.381999999999998 + - type: ndcg_at_3 + value: 12.469 + - type: ndcg_at_5 + value: 14.572 + - type: precision_at_1 + value: 8.488 + - type: precision_at_10 + value: 3.314 + - type: precision_at_100 + value: 0.674 + - type: precision_at_1000 + value: 0.093 + - type: precision_at_20 + value: 2.176 + - type: precision_at_3 + value: 6.016 + - type: precision_at_5 + value: 4.7620000000000005 + - type: recall_at_1 + value: 7.292999999999999 + - type: recall_at_10 + value: 28.109 + - type: recall_at_100 + value: 56.105000000000004 + - type: recall_at_1000 + value: 77.033 + - type: recall_at_20 + value: 36.447 + - type: recall_at_3 + value: 15.486 + - type: recall_at_5 + value: 20.471 + task: + type: Retrieval + - dataset: + config: default + name: MTEB QuoraRetrieval (default) + revision: e4e08e0b7dbe3c8700f0daef558ff32256715259 + split: test + type: mteb/quora + metrics: + - type: main_score + value: 78.944 + - type: map_at_1 + value: 62.163000000000004 + - type: map_at_10 + value: 74.406 + - type: map_at_100 + value: 75.211 + - type: map_at_1000 + value: 75.246 + - type: map_at_20 + value: 74.901 + - type: map_at_3 + value: 71.544 + - type: map_at_5 + value: 73.237 + - type: mrr_at_1 + value: 71.39999999999999 + - type: mrr_at_10 + value: 78.53174999999968 + - type: mrr_at_100 + value: 78.81864010375182 + - type: mrr_at_1000 + value: 78.82645374918509 + - type: mrr_at_20 + value: 78.71598608095506 + - type: mrr_at_3 + value: 77.11166666666631 + - type: mrr_at_5 + value: 77.97816666666611 + - type: nauc_map_at_1000_diff1 + value: 71.12026097344402 + - type: nauc_map_at_1000_max + value: 38.34659709478082 + - type: nauc_map_at_1000_std + value: -17.076551222440084 + - type: nauc_map_at_100_diff1 + value: 71.12970535718793 + - type: nauc_map_at_100_max + value: 38.34136449954092 + - type: nauc_map_at_100_std + value: -17.090201363548733 + - type: nauc_map_at_10_diff1 + value: 71.1639091984701 + - type: nauc_map_at_10_max + value: 37.92962545307399 + - type: nauc_map_at_10_std + value: -17.935251452443836 + - type: nauc_map_at_1_diff1 + value: 73.90119279220835 + - type: nauc_map_at_1_max + value: 30.96159752854218 + - type: nauc_map_at_1_std + value: -17.71968107861217 + - type: nauc_map_at_20_diff1 + value: 71.14144425695068 + - type: nauc_map_at_20_max + value: 38.24084182045044 + - type: nauc_map_at_20_std + value: -17.41338269274242 + - type: nauc_map_at_3_diff1 + value: 71.45595546693525 + - type: nauc_map_at_3_max + value: 36.08464694333641 + - type: nauc_map_at_3_std + value: -18.931004663754646 + - type: nauc_map_at_5_diff1 + value: 71.27464789828589 + - type: nauc_map_at_5_max + value: 37.30423556122988 + - type: nauc_map_at_5_std + value: -18.486579027688897 + - type: nauc_mrr_at_1000_diff1 + value: 71.9772870260667 + - type: nauc_mrr_at_1000_max + value: 40.84763399716098 + - type: nauc_mrr_at_1000_std + value: -14.825731250421617 + - type: nauc_mrr_at_100_diff1 + value: 71.97627495514737 + - type: nauc_mrr_at_100_max + value: 40.85239610393123 + - type: nauc_mrr_at_100_std + value: -14.81452787723373 + - type: nauc_mrr_at_10_diff1 + value: 71.88135293938872 + - type: nauc_mrr_at_10_max + value: 40.88653429928447 + - type: nauc_mrr_at_10_std + value: -14.921871682409485 + - type: nauc_mrr_at_1_diff1 + value: 73.6972904270755 + - type: nauc_mrr_at_1_max + value: 39.32918667821438 + - type: nauc_mrr_at_1_std + value: -15.176326573460615 + - type: nauc_mrr_at_20_diff1 + value: 71.95077268893942 + - type: nauc_mrr_at_20_max + value: 40.89414294678139 + - type: nauc_mrr_at_20_std + value: -14.82041706681107 + - type: nauc_mrr_at_3_diff1 + value: 71.84102470725445 + - type: nauc_mrr_at_3_max + value: 40.55979190068784 + - type: nauc_mrr_at_3_std + value: -15.426013609275019 + - type: nauc_mrr_at_5_diff1 + value: 71.7891912460372 + - type: nauc_mrr_at_5_max + value: 40.80511612753408 + - type: nauc_mrr_at_5_std + value: -15.18799645334625 + - type: nauc_ndcg_at_1000_diff1 + value: 70.89670367344667 + - type: nauc_ndcg_at_1000_max + value: 40.17914305815258 + - type: nauc_ndcg_at_1000_std + value: -14.984614028845971 + - type: nauc_ndcg_at_100_diff1 + value: 70.92339913111417 + - type: nauc_ndcg_at_100_max + value: 40.257745467547174 + - type: nauc_ndcg_at_100_std + value: -14.611387935135872 + - type: nauc_ndcg_at_10_diff1 + value: 70.48756735324325 + - type: nauc_ndcg_at_10_max + value: 39.58487015963329 + - type: nauc_ndcg_at_10_std + value: -16.645804593646147 + - type: nauc_ndcg_at_1_diff1 + value: 73.5844028830802 + - type: nauc_ndcg_at_1_max + value: 39.68976954212264 + - type: nauc_ndcg_at_1_std + value: -15.031929688958312 + - type: nauc_ndcg_at_20_diff1 + value: 70.7029749983589 + - type: nauc_ndcg_at_20_max + value: 40.12481563061243 + - type: nauc_ndcg_at_20_std + value: -15.54606689841114 + - type: nauc_ndcg_at_3_diff1 + value: 70.35105717009881 + - type: nauc_ndcg_at_3_max + value: 38.32508008621408 + - type: nauc_ndcg_at_3_std + value: -17.360779639396362 + - type: nauc_ndcg_at_5_diff1 + value: 70.38583658821312 + - type: nauc_ndcg_at_5_max + value: 39.01963672520352 + - type: nauc_ndcg_at_5_std + value: -17.239959123518233 + - type: nauc_precision_at_1000_diff1 + value: -36.21474583612648 + - type: nauc_precision_at_1000_max + value: -3.597428898937475 + - type: nauc_precision_at_1000_std + value: 20.250426782696966 + - type: nauc_precision_at_100_diff1 + value: -32.33501144245241 + - type: nauc_precision_at_100_max + value: -0.20640880335901107 + - type: nauc_precision_at_100_std + value: 20.25734872492559 + - type: nauc_precision_at_10_diff1 + value: -16.379871004767345 + - type: nauc_precision_at_10_max + value: 10.980061890327866 + - type: nauc_precision_at_10_std + value: 10.465941748223376 + - type: nauc_precision_at_1_diff1 + value: 73.5844028830802 + - type: nauc_precision_at_1_max + value: 39.68976954212264 + - type: nauc_precision_at_1_std + value: -15.031929688958312 + - type: nauc_precision_at_20_diff1 + value: -23.965337240125926 + - type: nauc_precision_at_20_max + value: 6.596649568590301 + - type: nauc_precision_at_20_std + value: 15.582400216847327 + - type: nauc_precision_at_3_diff1 + value: 10.489396277777569 + - type: nauc_precision_at_3_max + value: 21.625923996482737 + - type: nauc_precision_at_3_std + value: -0.7788020451085538 + - type: nauc_precision_at_5_diff1 + value: -3.737454839127393 + - type: nauc_precision_at_5_max + value: 16.933111460137404 + - type: nauc_precision_at_5_std + value: 4.7360335395138895 + - type: nauc_recall_at_1000_diff1 + value: 63.20571064867851 + - type: nauc_recall_at_1000_max + value: 57.229334552429314 + - type: nauc_recall_at_1000_std + value: 44.21186400138876 + - type: nauc_recall_at_100_diff1 + value: 64.53870069947862 + - type: nauc_recall_at_100_max + value: 47.10781661935386 + - type: nauc_recall_at_100_std + value: 16.521676724908925 + - type: nauc_recall_at_10_diff1 + value: 63.869924515916765 + - type: nauc_recall_at_10_max + value: 39.50325719990688 + - type: nauc_recall_at_10_std + value: -16.071467891980763 + - type: nauc_recall_at_1_diff1 + value: 73.90119279220835 + - type: nauc_recall_at_1_max + value: 30.96159752854218 + - type: nauc_recall_at_1_std + value: -17.71968107861217 + - type: nauc_recall_at_20_diff1 + value: 63.20541507276648 + - type: nauc_recall_at_20_max + value: 42.52369355095013 + - type: nauc_recall_at_20_std + value: -7.966841748999692 + - type: nauc_recall_at_3_diff1 + value: 66.95598825053953 + - type: nauc_recall_at_3_max + value: 34.813833345538 + - type: nauc_recall_at_3_std + value: -20.055997535188826 + - type: nauc_recall_at_5_diff1 + value: 65.4441887294711 + - type: nauc_recall_at_5_max + value: 37.18383133639401 + - type: nauc_recall_at_5_std + value: -19.04229973267749 + - type: ndcg_at_1 + value: 71.46000000000001 + - type: ndcg_at_10 + value: 78.944 + - type: ndcg_at_100 + value: 81.27799999999999 + - type: ndcg_at_1000 + value: 81.72500000000001 + - type: ndcg_at_20 + value: 80.041 + - type: ndcg_at_3 + value: 75.47 + - type: ndcg_at_5 + value: 77.181 + - type: precision_at_1 + value: 71.46000000000001 + - type: precision_at_10 + value: 11.935 + - type: precision_at_100 + value: 1.442 + - type: precision_at_1000 + value: 0.154 + - type: precision_at_20 + value: 6.444 + - type: precision_at_3 + value: 32.707 + - type: precision_at_5 + value: 21.592 + - type: recall_at_1 + value: 62.163000000000004 + - type: recall_at_10 + value: 87.69200000000001 + - type: recall_at_100 + value: 96.639 + - type: recall_at_1000 + value: 99.297 + - type: recall_at_20 + value: 91.401 + - type: recall_at_3 + value: 77.594 + - type: recall_at_5 + value: 82.431 + task: + type: Retrieval + - dataset: + config: default + name: MTEB RedditClustering (default) + revision: 24640382cdbf8abc73003fb0fa6d111a705499eb + split: test + type: mteb/reddit-clustering + metrics: + - type: main_score + value: 37.24085606808143 + - type: v_measure + value: 37.24085606808143 + - type: v_measure_std + value: 4.75871654527573 + task: + type: Clustering + - dataset: + config: default + name: MTEB RedditClusteringP2P (default) + revision: 385e3cb46b4cfa89021f56c4380204149d0efe33 + split: test + type: mteb/reddit-clustering-p2p + metrics: + - type: main_score + value: 44.53294634700315 + - type: v_measure + value: 44.53294634700315 + - type: v_measure_std + value: 11.281270481309583 + task: + type: Clustering + - dataset: + config: default + name: MTEB SCIDOCS (default) + revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88 + split: test + type: mteb/scidocs + metrics: + - type: main_score + value: 10.975999999999999 + - type: map_at_1 + value: 2.6229999999999998 + - type: map_at_10 + value: 6.043 + - type: map_at_100 + value: 7.274 + - type: map_at_1000 + value: 7.518 + - type: map_at_20 + value: 6.625 + - type: map_at_3 + value: 4.513 + - type: map_at_5 + value: 5.297000000000001 + - type: mrr_at_1 + value: 12.9 + - type: mrr_at_10 + value: 20.29456349206347 + - type: mrr_at_100 + value: 21.40608779578154 + - type: mrr_at_1000 + value: 21.5096323900278 + - type: mrr_at_20 + value: 20.89359279695191 + - type: mrr_at_3 + value: 17.9 + - type: mrr_at_5 + value: 19.264999999999986 + - type: nauc_map_at_1000_diff1 + value: 13.973997796384928 + - type: nauc_map_at_1000_max + value: 15.24396457979008 + - type: nauc_map_at_1000_std + value: 19.956152939934125 + - type: nauc_map_at_100_diff1 + value: 13.914326675244757 + - type: nauc_map_at_100_max + value: 14.992423212939576 + - type: nauc_map_at_100_std + value: 19.346582724513087 + - type: nauc_map_at_10_diff1 + value: 14.005203145683373 + - type: nauc_map_at_10_max + value: 12.533617494842156 + - type: nauc_map_at_10_std + value: 16.019921073048483 + - type: nauc_map_at_1_diff1 + value: 20.287526221402405 + - type: nauc_map_at_1_max + value: 12.29569598685881 + - type: nauc_map_at_1_std + value: 11.84440966885772 + - type: nauc_map_at_20_diff1 + value: 14.506149052503625 + - type: nauc_map_at_20_max + value: 14.128710240967395 + - type: nauc_map_at_20_std + value: 17.66552816835646 + - type: nauc_map_at_3_diff1 + value: 14.723437097271452 + - type: nauc_map_at_3_max + value: 13.97967494912635 + - type: nauc_map_at_3_std + value: 13.986282784554623 + - type: nauc_map_at_5_diff1 + value: 15.281528000748432 + - type: nauc_map_at_5_max + value: 12.74789727694775 + - type: nauc_map_at_5_std + value: 14.152734564652567 + - type: nauc_mrr_at_1000_diff1 + value: 14.909058242971323 + - type: nauc_mrr_at_1000_max + value: 14.89923029131018 + - type: nauc_mrr_at_1000_std + value: 16.112438519165448 + - type: nauc_mrr_at_100_diff1 + value: 14.921170730270145 + - type: nauc_mrr_at_100_max + value: 14.903295271302024 + - type: nauc_mrr_at_100_std + value: 16.142605858051482 + - type: nauc_mrr_at_10_diff1 + value: 15.055715022956981 + - type: nauc_mrr_at_10_max + value: 14.46564416594299 + - type: nauc_mrr_at_10_std + value: 15.462553449948851 + - type: nauc_mrr_at_1_diff1 + value: 20.38884124229307 + - type: nauc_mrr_at_1_max + value: 12.334690698559113 + - type: nauc_mrr_at_1_std + value: 12.311505189727232 + - type: nauc_mrr_at_20_diff1 + value: 14.906423936329604 + - type: nauc_mrr_at_20_max + value: 14.96346252727948 + - type: nauc_mrr_at_20_std + value: 15.957553065480726 + - type: nauc_mrr_at_3_diff1 + value: 14.84506515179406 + - type: nauc_mrr_at_3_max + value: 14.122529963628732 + - type: nauc_mrr_at_3_std + value: 15.038251505547425 + - type: nauc_mrr_at_5_diff1 + value: 14.761637229470903 + - type: nauc_mrr_at_5_max + value: 14.058545032558289 + - type: nauc_mrr_at_5_std + value: 15.257952718028047 + - type: nauc_ndcg_at_1000_diff1 + value: 12.401557025097095 + - type: nauc_ndcg_at_1000_max + value: 19.162281715396258 + - type: nauc_ndcg_at_1000_std + value: 27.28208364614879 + - type: nauc_ndcg_at_100_diff1 + value: 12.462531433192177 + - type: nauc_ndcg_at_100_max + value: 18.092239924308924 + - type: nauc_ndcg_at_100_std + value: 24.306771626400064 + - type: nauc_ndcg_at_10_diff1 + value: 13.012854438802716 + - type: nauc_ndcg_at_10_max + value: 13.240961212392408 + - type: nauc_ndcg_at_10_std + value: 17.16444103531682 + - type: nauc_ndcg_at_1_diff1 + value: 20.38884124229307 + - type: nauc_ndcg_at_1_max + value: 12.334690698559113 + - type: nauc_ndcg_at_1_std + value: 12.311505189727232 + - type: nauc_ndcg_at_20_diff1 + value: 13.405022785618451 + - type: nauc_ndcg_at_20_max + value: 16.129038815158523 + - type: nauc_ndcg_at_20_std + value: 20.13896063590036 + - type: nauc_ndcg_at_3_diff1 + value: 13.64223678386308 + - type: nauc_ndcg_at_3_max + value: 14.086175451408186 + - type: nauc_ndcg_at_3_std + value: 15.173192036409372 + - type: nauc_ndcg_at_5_diff1 + value: 14.075835104603799 + - type: nauc_ndcg_at_5_max + value: 12.991019756492042 + - type: nauc_ndcg_at_5_std + value: 15.083844651982483 + - type: nauc_precision_at_1000_diff1 + value: 6.679710695518377 + - type: nauc_precision_at_1000_max + value: 20.813353595075874 + - type: nauc_precision_at_1000_std + value: 36.13823850539846 + - type: nauc_precision_at_100_diff1 + value: 8.582899195791812 + - type: nauc_precision_at_100_max + value: 20.606190885640554 + - type: nauc_precision_at_100_std + value: 31.015207458735837 + - type: nauc_precision_at_10_diff1 + value: 9.955421784523907 + - type: nauc_precision_at_10_max + value: 12.226547316835655 + - type: nauc_precision_at_10_std + value: 19.194260501366102 + - type: nauc_precision_at_1_diff1 + value: 20.38884124229307 + - type: nauc_precision_at_1_max + value: 12.334690698559113 + - type: nauc_precision_at_1_std + value: 12.311505189727232 + - type: nauc_precision_at_20_diff1 + value: 10.821274632720215 + - type: nauc_precision_at_20_max + value: 17.910630241041957 + - type: nauc_precision_at_20_std + value: 24.089477303794364 + - type: nauc_precision_at_3_diff1 + value: 11.280609220738322 + - type: nauc_precision_at_3_max + value: 14.000592735288128 + - type: nauc_precision_at_3_std + value: 16.066715135125115 + - type: nauc_precision_at_5_diff1 + value: 11.973988845160886 + - type: nauc_precision_at_5_max + value: 11.63579702369595 + - type: nauc_precision_at_5_std + value: 15.66853420805326 + - type: nauc_recall_at_1000_diff1 + value: 7.236848009802737 + - type: nauc_recall_at_1000_max + value: 21.90002660793917 + - type: nauc_recall_at_1000_std + value: 36.73039901613978 + - type: nauc_recall_at_100_diff1 + value: 8.781121548109736 + - type: nauc_recall_at_100_max + value: 20.872807676915585 + - type: nauc_recall_at_100_std + value: 31.089252844325383 + - type: nauc_recall_at_10_diff1 + value: 10.403929936534533 + - type: nauc_recall_at_10_max + value: 12.317583820258339 + - type: nauc_recall_at_10_std + value: 18.87239766836694 + - type: nauc_recall_at_1_diff1 + value: 20.287526221402405 + - type: nauc_recall_at_1_max + value: 12.29569598685881 + - type: nauc_recall_at_1_std + value: 11.84440966885772 + - type: nauc_recall_at_20_diff1 + value: 11.053723609378533 + - type: nauc_recall_at_20_max + value: 17.904832495199997 + - type: nauc_recall_at_20_std + value: 23.94487620259356 + - type: nauc_recall_at_3_diff1 + value: 11.448063932383073 + - type: nauc_recall_at_3_max + value: 14.030673949295828 + - type: nauc_recall_at_3_std + value: 15.707862370994691 + - type: nauc_recall_at_5_diff1 + value: 12.254302630244073 + - type: nauc_recall_at_5_max + value: 11.713720712309172 + - type: nauc_recall_at_5_std + value: 15.337816546798894 + - type: ndcg_at_1 + value: 12.9 + - type: ndcg_at_10 + value: 10.975999999999999 + - type: ndcg_at_100 + value: 16.808 + - type: ndcg_at_1000 + value: 22.061 + - type: ndcg_at_20 + value: 12.854 + - type: ndcg_at_3 + value: 10.544 + - type: ndcg_at_5 + value: 9.200999999999999 + - type: precision_at_1 + value: 12.9 + - type: precision_at_10 + value: 5.66 + - type: precision_at_100 + value: 1.422 + - type: precision_at_1000 + value: 0.269 + - type: precision_at_20 + value: 3.925 + - type: precision_at_3 + value: 9.866999999999999 + - type: precision_at_5 + value: 8.08 + - type: recall_at_1 + value: 2.6229999999999998 + - type: recall_at_10 + value: 11.458 + - type: recall_at_100 + value: 28.852 + - type: recall_at_1000 + value: 54.65 + - type: recall_at_20 + value: 15.906999999999998 + - type: recall_at_3 + value: 6.008 + - type: recall_at_5 + value: 8.187999999999999 + task: + type: Retrieval + - dataset: + config: default + name: MTEB SICK-R (default) + revision: 20a6d6f312dd54037fe07a32d58e5e168867909d + split: test + type: mteb/sickr-sts + metrics: + - type: cosine_pearson + value: 75.3765633045698 + - type: cosine_spearman + value: 63.98613448451591 + - type: euclidean_pearson + value: 68.77492302556632 + - type: euclidean_spearman + value: 63.98600896235449 + - type: main_score + value: 63.98613448451591 + - type: manhattan_pearson + value: 68.97719158550568 + - type: manhattan_spearman + value: 64.17632002019486 + - type: pearson + value: 75.3765633045698 + - type: spearman + value: 63.98613448451591 + task: + type: STS + - dataset: + config: default + name: MTEB STS12 (default) + revision: a0d554a64d88156834ff5ae9920b964011b16384 + split: test + type: mteb/sts12-sts + metrics: + - type: cosine_pearson + value: 72.15434642732643 + - type: cosine_spearman + value: 62.37621006422606 + - type: euclidean_pearson + value: 68.4217723007459 + - type: euclidean_spearman + value: 62.3761516548255 + - type: main_score + value: 62.37621006422606 + - type: manhattan_pearson + value: 68.44462184344337 + - type: manhattan_spearman + value: 62.409862632343504 + - type: pearson + value: 72.15434642732643 + - type: spearman + value: 62.37621006422606 + task: + type: STS + - dataset: + config: default + name: MTEB STS13 (default) + revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca + split: test + type: mteb/sts13-sts + metrics: + - type: cosine_pearson + value: 75.8217088748812 + - type: cosine_spearman + value: 76.81750749576634 + - type: euclidean_pearson + value: 76.28374455431411 + - type: euclidean_spearman + value: 76.81750749576634 + - type: main_score + value: 76.81750749576634 + - type: manhattan_pearson + value: 76.22730912471911 + - type: manhattan_spearman + value: 76.76337659370503 + - type: pearson + value: 75.8217088748812 + - type: spearman + value: 76.81750749576634 + task: + type: STS + - dataset: + config: default + name: MTEB STS14 (default) + revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375 + split: test + type: mteb/sts14-sts + metrics: + - type: cosine_pearson + value: 75.44045207503216 + - type: cosine_spearman + value: 70.98200491385815 + - type: euclidean_pearson + value: 73.79528647452996 + - type: euclidean_spearman + value: 70.98200491385815 + - type: main_score + value: 70.98200491385815 + - type: manhattan_pearson + value: 74.00347105187919 + - type: manhattan_spearman + value: 71.15597425184266 + - type: pearson + value: 75.44045207503216 + - type: spearman + value: 70.98200491385815 + task: + type: STS + - dataset: + config: default + name: MTEB STS15 (default) + revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3 + split: test + type: mteb/sts15-sts + metrics: + - type: cosine_pearson + value: 77.94210244961548 + - type: cosine_spearman + value: 78.66432028882325 + - type: euclidean_pearson + value: 78.35289467379204 + - type: euclidean_spearman + value: 78.66432028882325 + - type: main_score + value: 78.66432028882325 + - type: manhattan_pearson + value: 78.28424867038663 + - type: manhattan_spearman + value: 78.58719353431712 + - type: pearson + value: 77.94210244961548 + - type: spearman + value: 78.66432028882325 + task: + type: STS + - dataset: + config: default + name: MTEB STS16 (default) + revision: 4d8694f8f0e0100860b497b999b3dbed754a0513 + split: test + type: mteb/sts16-sts + metrics: + - type: cosine_pearson + value: 74.09988267613177 + - type: cosine_spearman + value: 75.09842774068727 + - type: euclidean_pearson + value: 74.68409526331901 + - type: euclidean_spearman + value: 75.09842774068727 + - type: main_score + value: 75.09842774068727 + - type: manhattan_pearson + value: 74.59031379824495 + - type: manhattan_spearman + value: 75.02693979965505 + - type: pearson + value: 74.09988267613177 + - type: spearman + value: 75.09842774068727 + task: + type: STS + - dataset: + config: en-tr + name: MTEB STS17 (en-tr) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 14.399972034167586 + - type: cosine_spearman + value: 11.974036233167949 + - type: euclidean_pearson + value: 14.485293885486817 + - type: euclidean_spearman + value: 11.974036233167949 + - type: main_score + value: 11.974036233167949 + - type: manhattan_pearson + value: 13.719281545868167 + - type: manhattan_spearman + value: 11.030737066771716 + - type: pearson + value: 14.399972034167586 + - type: spearman + value: 11.974036233167949 + task: + type: STS + - dataset: + config: en-ar + name: MTEB STS17 (en-ar) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 0.48735904843064204 + - type: cosine_spearman + value: -0.5586853869425744 + - type: euclidean_pearson + value: 0.6087049823875109 + - type: euclidean_spearman + value: -0.5586853869425744 + - type: main_score + value: -0.5586853869425744 + - type: manhattan_pearson + value: -5.604629753019841 + - type: manhattan_spearman + value: -6.156533065413701 + - type: pearson + value: 0.48735904843064204 + - type: spearman + value: -0.5586853869425744 + task: + type: STS + - dataset: + config: en-en + name: MTEB STS17 (en-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 83.08779566633636 + - type: cosine_spearman + value: 84.49159199501017 + - type: euclidean_pearson + value: 83.5601221620653 + - type: euclidean_spearman + value: 84.49159199501017 + - type: main_score + value: 84.49159199501017 + - type: manhattan_pearson + value: 83.66974922388209 + - type: manhattan_spearman + value: 84.69728258948982 + - type: pearson + value: 83.08779566633636 + - type: spearman + value: 84.49159199501017 + task: + type: STS + - dataset: + config: nl-en + name: MTEB STS17 (nl-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 28.253544786263518 + - type: cosine_spearman + value: 24.619138389390237 + - type: euclidean_pearson + value: 28.492064397709616 + - type: euclidean_spearman + value: 24.619138389390237 + - type: main_score + value: 24.619138389390237 + - type: manhattan_pearson + value: 30.135975855905052 + - type: manhattan_spearman + value: 25.508286051679313 + - type: pearson + value: 28.253544786263518 + - type: spearman + value: 24.619138389390237 + task: + type: STS + - dataset: + config: fr-en + name: MTEB STS17 (fr-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 32.599578555181466 + - type: cosine_spearman + value: 31.33358514527386 + - type: euclidean_pearson + value: 32.613214551816796 + - type: euclidean_spearman + value: 31.33358514527386 + - type: main_score + value: 31.33358514527386 + - type: manhattan_pearson + value: 32.25686415499878 + - type: manhattan_spearman + value: 29.668763682572735 + - type: pearson + value: 32.599578555181466 + - type: spearman + value: 31.33358514527386 + task: + type: STS + - dataset: + config: it-en + name: MTEB STS17 (it-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 23.75577872956453 + - type: cosine_spearman + value: 21.321554136347544 + - type: euclidean_pearson + value: 23.28498966277531 + - type: euclidean_spearman + value: 21.321554136347544 + - type: main_score + value: 21.321554136347544 + - type: manhattan_pearson + value: 21.83738887582424 + - type: manhattan_spearman + value: 18.096121115941422 + - type: pearson + value: 23.75577872956453 + - type: spearman + value: 21.321554136347544 + task: + type: STS + - dataset: + config: en-de + name: MTEB STS17 (en-de) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 27.69563703760355 + - type: cosine_spearman + value: 24.76336397535317 + - type: euclidean_pearson + value: 28.070464936235396 + - type: euclidean_spearman + value: 24.76336397535317 + - type: main_score + value: 24.76336397535317 + - type: manhattan_pearson + value: 29.820923974737546 + - type: manhattan_spearman + value: 27.812820254677657 + - type: pearson + value: 27.69563703760355 + - type: spearman + value: 24.76336397535317 + task: + type: STS + - dataset: + config: es-en + name: MTEB STS17 (es-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 16.346177540651276 + - type: cosine_spearman + value: 13.601344161361006 + - type: euclidean_pearson + value: 16.494437424874043 + - type: euclidean_spearman + value: 13.601344161361006 + - type: main_score + value: 13.601344161361006 + - type: manhattan_pearson + value: 14.109457511241242 + - type: manhattan_spearman + value: 13.157710082543057 + - type: pearson + value: 16.346177540651276 + - type: spearman + value: 13.601344161361006 + task: + type: STS + - dataset: + config: pl-en + name: MTEB STS22 (pl-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 8.721258908976726 + - type: cosine_spearman + value: 18.35245351814682 + - type: euclidean_pearson + value: 6.733168292100376 + - type: euclidean_spearman + value: 18.35245351814682 + - type: main_score + value: 18.35245351814682 + - type: manhattan_pearson + value: 10.48470979974683 + - type: manhattan_spearman + value: 23.301312684266456 + - type: pearson + value: 8.721258908976726 + - type: spearman + value: 18.35245351814682 + task: + type: STS + - dataset: + config: zh-en + name: MTEB STS22 (zh-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 11.49126181142697 + - type: cosine_spearman + value: 16.21762340764813 + - type: euclidean_pearson + value: 10.421413269548768 + - type: euclidean_spearman + value: 16.15075860419545 + - type: main_score + value: 16.21762340764813 + - type: manhattan_pearson + value: 14.305549482593522 + - type: manhattan_spearman + value: 18.371358872490433 + - type: pearson + value: 11.49126181142697 + - type: spearman + value: 16.21762340764813 + task: + type: STS + - dataset: + config: de-en + name: MTEB STS22 (de-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 26.168037910219216 + - type: cosine_spearman + value: 31.68384203851073 + - type: euclidean_pearson + value: 25.745626390764293 + - type: euclidean_spearman + value: 31.677361748737955 + - type: main_score + value: 31.68384203851073 + - type: manhattan_pearson + value: 25.87013667642989 + - type: manhattan_spearman + value: 34.71610605034525 + - type: pearson + value: 26.168037910219216 + - type: spearman + value: 31.68384203851073 + task: + type: STS + - dataset: + config: es-en + name: MTEB STS22 (es-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 9.071484790364082 + - type: cosine_spearman + value: 9.726488833098573 + - type: euclidean_pearson + value: 8.563790512620805 + - type: euclidean_spearman + value: 9.741694852789207 + - type: main_score + value: 9.726488833098573 + - type: manhattan_pearson + value: 6.653824957307301 + - type: manhattan_spearman + value: 7.550086762351419 + - type: pearson + value: 9.071484790364082 + - type: spearman + value: 9.726488833098573 + task: + type: STS + - dataset: + config: en + name: MTEB STS22 (en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 58.835618895152855 + - type: cosine_spearman + value: 63.35773898104199 + - type: euclidean_pearson + value: 62.348443804922525 + - type: euclidean_spearman + value: 63.35773898104199 + - type: main_score + value: 63.35773898104199 + - type: manhattan_pearson + value: 62.7037835433589 + - type: manhattan_spearman + value: 62.86339756370198 + - type: pearson + value: 58.835618895152855 + - type: spearman + value: 63.35773898104199 + task: + type: STS + - dataset: + config: default + name: MTEB STSBenchmark (default) + revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831 + split: test + type: mteb/stsbenchmark-sts + metrics: + - type: cosine_pearson + value: 75.85762627227177 + - type: cosine_spearman + value: 74.28444387078392 + - type: euclidean_pearson + value: 76.14631470926992 + - type: euclidean_spearman + value: 74.28444387078392 + - type: main_score + value: 74.28444387078392 + - type: manhattan_pearson + value: 76.29026607759047 + - type: manhattan_spearman + value: 74.36833261994545 + - type: pearson + value: 75.85762627227177 + - type: spearman + value: 74.28444387078392 + task: + type: STS + - dataset: + config: default + name: MTEB SciDocsRR (default) + revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab + split: test + type: mteb/scidocs-reranking + metrics: + - type: main_score + value: 70.28519419296406 + - type: map + value: 70.28519419296406 + - type: mrr + value: 89.42476278260591 + - type: nAUC_map_diff1 + value: 9.174758618691703 + - type: nAUC_map_max + value: 56.09564435538077 + - type: nAUC_map_std + value: 67.85518368829582 + - type: nAUC_mrr_diff1 + value: 43.619263369174874 + - type: nAUC_mrr_max + value: 74.99462642243834 + - type: nAUC_mrr_std + value: 71.03708831031823 + task: + type: Reranking + - dataset: + config: default + name: MTEB SciFact (default) + revision: 0228b52cf27578f30900b9e5271d331663a030d7 + split: test + type: mteb/scifact + metrics: + - type: main_score + value: 46.039 + - type: map_at_1 + value: 32.833 + - type: map_at_10 + value: 41.073 + - type: map_at_100 + value: 42.055 + - type: map_at_1000 + value: 42.115 + - type: map_at_20 + value: 41.577 + - type: map_at_3 + value: 38.181 + - type: map_at_5 + value: 39.939 + - type: mrr_at_1 + value: 34.66666666666667 + - type: mrr_at_10 + value: 42.69973544973544 + - type: mrr_at_100 + value: 43.50094990516196 + - type: mrr_at_1000 + value: 43.55518326135714 + - type: mrr_at_20 + value: 43.09661278264219 + - type: mrr_at_3 + value: 40.055555555555536 + - type: mrr_at_5 + value: 41.755555555555546 + - type: nauc_map_at_1000_diff1 + value: 45.428609740785745 + - type: nauc_map_at_1000_max + value: 37.543057127562015 + - type: nauc_map_at_1000_std + value: 2.996790610257377 + - type: nauc_map_at_100_diff1 + value: 45.41655665365538 + - type: nauc_map_at_100_max + value: 37.55142170311717 + - type: nauc_map_at_100_std + value: 3.021634986612468 + - type: nauc_map_at_10_diff1 + value: 45.28151352698938 + - type: nauc_map_at_10_max + value: 37.15161941800589 + - type: nauc_map_at_10_std + value: 2.1466882062165955 + - type: nauc_map_at_1_diff1 + value: 46.886698245684144 + - type: nauc_map_at_1_max + value: 34.23436427795755 + - type: nauc_map_at_1_std + value: -1.63247086935568 + - type: nauc_map_at_20_diff1 + value: 45.323443483378334 + - type: nauc_map_at_20_max + value: 37.360033627177806 + - type: nauc_map_at_20_std + value: 2.5814764857730226 + - type: nauc_map_at_3_diff1 + value: 46.42738772300192 + - type: nauc_map_at_3_max + value: 34.91781909311709 + - type: nauc_map_at_3_std + value: 0.7407858981306185 + - type: nauc_map_at_5_diff1 + value: 45.56095939652318 + - type: nauc_map_at_5_max + value: 36.42349927345809 + - type: nauc_map_at_5_std + value: 1.5882842990321877 + - type: nauc_mrr_at_1000_diff1 + value: 46.619194210193115 + - type: nauc_mrr_at_1000_max + value: 39.58593846571098 + - type: nauc_mrr_at_1000_std + value: 6.871860120606581 + - type: nauc_mrr_at_100_diff1 + value: 46.60992557885071 + - type: nauc_mrr_at_100_max + value: 39.59384195719456 + - type: nauc_mrr_at_100_std + value: 6.913799322671517 + - type: nauc_mrr_at_10_diff1 + value: 46.49797118603073 + - type: nauc_mrr_at_10_max + value: 39.50640104704752 + - type: nauc_mrr_at_10_std + value: 6.508999197821892 + - type: nauc_mrr_at_1_diff1 + value: 49.3185659952106 + - type: nauc_mrr_at_1_max + value: 36.96867516551628 + - type: nauc_mrr_at_1_std + value: 3.2209642203127 + - type: nauc_mrr_at_20_diff1 + value: 46.57642255869317 + - type: nauc_mrr_at_20_max + value: 39.57536672051077 + - type: nauc_mrr_at_20_std + value: 6.686866161626959 + - type: nauc_mrr_at_3_diff1 + value: 47.71472878192224 + - type: nauc_mrr_at_3_max + value: 37.84645732330119 + - type: nauc_mrr_at_3_std + value: 5.794681287411272 + - type: nauc_mrr_at_5_diff1 + value: 46.88419733444017 + - type: nauc_mrr_at_5_max + value: 38.93781880268596 + - type: nauc_mrr_at_5_std + value: 5.995326077656308 + - type: nauc_ndcg_at_1000_diff1 + value: 44.62859803359441 + - type: nauc_ndcg_at_1000_max + value: 40.04051001437065 + - type: nauc_ndcg_at_1000_std + value: 7.077880529469134 + - type: nauc_ndcg_at_100_diff1 + value: 44.27396584172457 + - type: nauc_ndcg_at_100_max + value: 40.45032063067928 + - type: nauc_ndcg_at_100_std + value: 8.295284232687681 + - type: nauc_ndcg_at_10_diff1 + value: 44.12502453790537 + - type: nauc_ndcg_at_10_max + value: 39.397967545534506 + - type: nauc_ndcg_at_10_std + value: 5.154810286641697 + - type: nauc_ndcg_at_1_diff1 + value: 49.3185659952106 + - type: nauc_ndcg_at_1_max + value: 36.96867516551628 + - type: nauc_ndcg_at_1_std + value: 3.2209642203127 + - type: nauc_ndcg_at_20_diff1 + value: 44.36600328826306 + - type: nauc_ndcg_at_20_max + value: 39.81785860876702 + - type: nauc_ndcg_at_20_std + value: 6.377952248139375 + - type: nauc_ndcg_at_3_diff1 + value: 46.48606615048381 + - type: nauc_ndcg_at_3_max + value: 35.70009304999929 + - type: nauc_ndcg_at_3_std + value: 3.154595246639392 + - type: nauc_ndcg_at_5_diff1 + value: 44.904178071715464 + - type: nauc_ndcg_at_5_max + value: 37.949117517935456 + - type: nauc_ndcg_at_5_std + value: 3.8124217549794093 + - type: nauc_precision_at_1000_diff1 + value: -0.8300581285183218 + - type: nauc_precision_at_1000_max + value: 31.934010990018514 + - type: nauc_precision_at_1000_std + value: 47.92943894671857 + - type: nauc_precision_at_100_diff1 + value: 14.358545370047567 + - type: nauc_precision_at_100_max + value: 43.205762751248905 + - type: nauc_precision_at_100_std + value: 44.94248867201113 + - type: nauc_precision_at_10_diff1 + value: 32.758240141259556 + - type: nauc_precision_at_10_max + value: 46.157053168530545 + - type: nauc_precision_at_10_std + value: 21.028300961349817 + - type: nauc_precision_at_1_diff1 + value: 49.3185659952106 + - type: nauc_precision_at_1_max + value: 36.96867516551628 + - type: nauc_precision_at_1_std + value: 3.2209642203127 + - type: nauc_precision_at_20_diff1 + value: 29.999274269632952 + - type: nauc_precision_at_20_max + value: 46.11250433422294 + - type: nauc_precision_at_20_std + value: 27.805552643674424 + - type: nauc_precision_at_3_diff1 + value: 44.76807259977309 + - type: nauc_precision_at_3_max + value: 39.34948953284521 + - type: nauc_precision_at_3_std + value: 12.711560276374097 + - type: nauc_precision_at_5_diff1 + value: 39.43498516172891 + - type: nauc_precision_at_5_max + value: 44.48289868527945 + - type: nauc_precision_at_5_std + value: 16.414112107697655 + - type: nauc_recall_at_1000_diff1 + value: 26.98339178190966 + - type: nauc_recall_at_1000_max + value: 65.3524927031179 + - type: nauc_recall_at_1000_std + value: 34.59530793944871 + - type: nauc_recall_at_100_diff1 + value: 33.663296786200256 + - type: nauc_recall_at_100_max + value: 53.06676443361575 + - type: nauc_recall_at_100_std + value: 35.88980433138387 + - type: nauc_recall_at_10_diff1 + value: 38.095698552275756 + - type: nauc_recall_at_10_max + value: 43.21866470802114 + - type: nauc_recall_at_10_std + value: 10.055255790665134 + - type: nauc_recall_at_1_diff1 + value: 46.886698245684144 + - type: nauc_recall_at_1_max + value: 34.23436427795755 + - type: nauc_recall_at_1_std + value: -1.63247086935568 + - type: nauc_recall_at_20_diff1 + value: 39.0968003991353 + - type: nauc_recall_at_20_max + value: 45.126688167134404 + - type: nauc_recall_at_20_std + value: 15.411347479962858 + - type: nauc_recall_at_3_diff1 + value: 45.02281186238651 + - type: nauc_recall_at_3_max + value: 33.51469893830599 + - type: nauc_recall_at_3_std + value: 4.0471017429782385 + - type: nauc_recall_at_5_diff1 + value: 40.702224348966396 + - type: nauc_recall_at_5_max + value: 38.8291162721674 + - type: nauc_recall_at_5_std + value: 5.599714100970187 + - type: ndcg_at_1 + value: 34.666999999999994 + - type: ndcg_at_10 + value: 46.039 + - type: ndcg_at_100 + value: 50.88 + - type: ndcg_at_1000 + value: 52.516 + - type: ndcg_at_20 + value: 47.615 + - type: ndcg_at_3 + value: 40.425 + - type: ndcg_at_5 + value: 43.444 + - type: precision_at_1 + value: 34.666999999999994 + - type: precision_at_10 + value: 6.6000000000000005 + - type: precision_at_100 + value: 0.9400000000000001 + - type: precision_at_1000 + value: 0.108 + - type: precision_at_20 + value: 3.6670000000000003 + - type: precision_at_3 + value: 16.111 + - type: precision_at_5 + value: 11.333 + - type: recall_at_1 + value: 32.833 + - type: recall_at_10 + value: 59.955999999999996 + - type: recall_at_100 + value: 82.89999999999999 + - type: recall_at_1000 + value: 95.767 + - type: recall_at_20 + value: 65.789 + - type: recall_at_3 + value: 44.806000000000004 + - type: recall_at_5 + value: 52.111 + task: + type: Retrieval + - dataset: + config: default + name: MTEB SprintDuplicateQuestions (default) + revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 + split: test + type: mteb/sprintduplicatequestions-pairclassification + metrics: + - type: cosine_accuracy + value: 99.71188118811881 + - type: cosine_accuracy_threshold + value: 80.28961679083551 + - type: cosine_ap + value: 89.87404753184795 + - type: cosine_f1 + value: 85.25088697415104 + - type: cosine_f1_threshold + value: 78.87657179254246 + - type: cosine_precision + value: 86.43371017471738 + - type: cosine_recall + value: 84.1 + - type: dot_accuracy + value: 99.71188118811881 + - type: dot_accuracy_threshold + value: 80.28961582816035 + - type: dot_ap + value: 89.87404753184795 + - type: dot_f1 + value: 85.25088697415104 + - type: dot_f1_threshold + value: 78.87657197624056 + - type: dot_precision + value: 86.43371017471738 + - type: dot_recall + value: 84.1 + - type: euclidean_accuracy + value: 99.71188118811881 + - type: euclidean_accuracy_threshold + value: 62.785946440274465 + - type: euclidean_ap + value: 89.87404753184795 + - type: euclidean_f1 + value: 85.25088697415104 + - type: euclidean_f1_threshold + value: 64.9975660049523 + - type: euclidean_precision + value: 86.43371017471738 + - type: euclidean_recall + value: 84.1 + - type: main_score + value: 89.87404753184795 + - type: manhattan_accuracy + value: 99.71782178217822 + - type: manhattan_accuracy_threshold + value: 571.5442430373514 + - type: manhattan_ap + value: 89.86667222138405 + - type: manhattan_f1 + value: 85.3017019082001 + - type: manhattan_f1_threshold + value: 571.5442430373514 + - type: manhattan_precision + value: 88.07241746538871 + - type: manhattan_recall + value: 82.69999999999999 + - type: max_accuracy + value: 99.71782178217822 + - type: max_ap + value: 89.87404753184795 + - type: max_f1 + value: 85.3017019082001 + - type: max_precision + value: 88.07241746538871 + - type: max_recall + value: 84.1 + - type: similarity_accuracy + value: 99.71188118811881 + - type: similarity_accuracy_threshold + value: 80.28961679083551 + - type: similarity_ap + value: 89.87404753184795 + - type: similarity_f1 + value: 85.25088697415104 + - type: similarity_f1_threshold + value: 78.87657179254246 + - type: similarity_precision + value: 86.43371017471738 + - type: similarity_recall + value: 84.1 + task: + type: PairClassification + - dataset: + config: default + name: MTEB StackExchangeClustering (default) + revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 + split: test + type: mteb/stackexchange-clustering + metrics: + - type: main_score + value: 45.28623588344632 + - type: v_measure + value: 45.28623588344632 + - type: v_measure_std + value: 3.699756512710844 + task: + type: Clustering + - dataset: + config: default + name: MTEB StackExchangeClusteringP2P (default) + revision: 815ca46b2622cec33ccafc3735d572c266efdb44 + split: test + type: mteb/stackexchange-clustering-p2p + metrics: + - type: main_score + value: 31.36188637863386 + - type: v_measure + value: 31.36188637863386 + - type: v_measure_std + value: 1.479842367330655 + task: + type: Clustering + - dataset: + config: default + name: MTEB StackOverflowDupQuestions (default) + revision: e185fbe320c72810689fc5848eb6114e1ef5ec69 + split: test + type: mteb/stackoverflowdupquestions-reranking + metrics: + - type: main_score + value: 41.90071199374392 + - type: map + value: 41.90071199374392 + - type: mrr + value: 42.24937133944486 + - type: nAUC_map_diff1 + value: 34.56201480309732 + - type: nAUC_map_max + value: 13.244556287676348 + - type: nAUC_map_std + value: 4.280183327551114 + - type: nAUC_mrr_diff1 + value: 33.70148851327892 + - type: nAUC_mrr_max + value: 14.254825331290972 + - type: nAUC_mrr_std + value: 4.529615444769097 + task: + type: Reranking + - dataset: + config: default + name: MTEB SummEval (default) + revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c + split: test + type: mteb/summeval + metrics: + - type: cosine_pearson + value: 30.390849690222034 + - type: cosine_spearman + value: 28.891662569282467 + - type: dot_pearson + value: 30.390849348187416 + - type: dot_spearman + value: 28.798804320984324 + - type: main_score + value: 28.891662569282467 + - type: pearson + value: 30.390849690222034 + - type: spearman + value: 28.891662569282467 + task: + type: Summarization + - dataset: + config: default + name: MTEB TRECCOVID (default) + revision: bb9466bac8153a0349341eb1b22e06409e78ef4e + split: test + type: mteb/trec-covid + metrics: + - type: main_score + value: 43.236000000000004 + - type: map_at_1 + value: 0.145 + - type: map_at_10 + value: 0.9209999999999999 + - type: map_at_100 + value: 4.718 + - type: map_at_1000 + value: 11.262 + - type: map_at_20 + value: 1.637 + - type: map_at_3 + value: 0.35000000000000003 + - type: map_at_5 + value: 0.532 + - type: mrr_at_1 + value: 62.0 + - type: mrr_at_10 + value: 69.40238095238095 + - type: mrr_at_100 + value: 69.55750915750914 + - type: mrr_at_1000 + value: 69.59003822688032 + - type: mrr_at_20 + value: 69.40238095238095 + - type: mrr_at_3 + value: 67.66666666666667 + - type: mrr_at_5 + value: 67.66666666666667 + - type: nauc_map_at_1000_diff1 + value: -0.9866281041682196 + - type: nauc_map_at_1000_max + value: 31.12085608035618 + - type: nauc_map_at_1000_std + value: 53.009123272875236 + - type: nauc_map_at_100_diff1 + value: -10.423991731052672 + - type: nauc_map_at_100_max + value: 21.253437633464 + - type: nauc_map_at_100_std + value: 28.44817556481647 + - type: nauc_map_at_10_diff1 + value: 0.9108503211560519 + - type: nauc_map_at_10_max + value: 24.561084063694974 + - type: nauc_map_at_10_std + value: 18.05662846072466 + - type: nauc_map_at_1_diff1 + value: 2.163025554719988 + - type: nauc_map_at_1_max + value: 4.173360349606284 + - type: nauc_map_at_1_std + value: 3.8151957830662955 + - type: nauc_map_at_20_diff1 + value: -3.5278217716407774 + - type: nauc_map_at_20_max + value: 21.530434946303384 + - type: nauc_map_at_20_std + value: 17.503504606889667 + - type: nauc_map_at_3_diff1 + value: -5.37288514415413 + - type: nauc_map_at_3_max + value: 19.159151414876334 + - type: nauc_map_at_3_std + value: 10.438553350197065 + - type: nauc_map_at_5_diff1 + value: 0.2040960199712178 + - type: nauc_map_at_5_max + value: 20.989057156573026 + - type: nauc_map_at_5_std + value: 12.31339487014647 + - type: nauc_mrr_at_1000_diff1 + value: 16.231982255141002 + - type: nauc_mrr_at_1000_max + value: 22.157032073031953 + - type: nauc_mrr_at_1000_std + value: 17.25244660114329 + - type: nauc_mrr_at_100_diff1 + value: 16.291634278945995 + - type: nauc_mrr_at_100_max + value: 22.111932462460885 + - type: nauc_mrr_at_100_std + value: 17.175937439333218 + - type: nauc_mrr_at_10_diff1 + value: 15.916829499802285 + - type: nauc_mrr_at_10_max + value: 22.02196974730754 + - type: nauc_mrr_at_10_std + value: 17.095811543787327 + - type: nauc_mrr_at_1_diff1 + value: 13.58643817493768 + - type: nauc_mrr_at_1_max + value: 21.816447712518837 + - type: nauc_mrr_at_1_std + value: 17.924499276989756 + - type: nauc_mrr_at_20_diff1 + value: 15.916829499802285 + - type: nauc_mrr_at_20_max + value: 22.02196974730754 + - type: nauc_mrr_at_20_std + value: 17.095811543787327 + - type: nauc_mrr_at_3_diff1 + value: 16.861922990056758 + - type: nauc_mrr_at_3_max + value: 23.67224734912763 + - type: nauc_mrr_at_3_std + value: 18.77182517851052 + - type: nauc_mrr_at_5_diff1 + value: 16.861922990056758 + - type: nauc_mrr_at_5_max + value: 23.67224734912763 + - type: nauc_mrr_at_5_std + value: 18.77182517851052 + - type: nauc_ndcg_at_1000_diff1 + value: -3.6322351354234277 + - type: nauc_ndcg_at_1000_max + value: 22.84735239467931 + - type: nauc_ndcg_at_1000_std + value: 47.101995953544375 + - type: nauc_ndcg_at_100_diff1 + value: 1.2976318687207455 + - type: nauc_ndcg_at_100_max + value: 25.603883117672627 + - type: nauc_ndcg_at_100_std + value: 43.82225311236483 + - type: nauc_ndcg_at_10_diff1 + value: 1.4385026341900742 + - type: nauc_ndcg_at_10_max + value: 26.983102845386192 + - type: nauc_ndcg_at_10_std + value: 30.974303592515234 + - type: nauc_ndcg_at_1_diff1 + value: 11.626777860284411 + - type: nauc_ndcg_at_1_max + value: 19.944879191180636 + - type: nauc_ndcg_at_1_std + value: 17.19740675158504 + - type: nauc_ndcg_at_20_diff1 + value: 0.7222731371858458 + - type: nauc_ndcg_at_20_max + value: 25.565239587362182 + - type: nauc_ndcg_at_20_std + value: 33.250012456509396 + - type: nauc_ndcg_at_3_diff1 + value: -0.048425941543918084 + - type: nauc_ndcg_at_3_max + value: 31.34942513461199 + - type: nauc_ndcg_at_3_std + value: 23.849711658390376 + - type: nauc_ndcg_at_5_diff1 + value: 2.7179781299699624 + - type: nauc_ndcg_at_5_max + value: 28.356065716200323 + - type: nauc_ndcg_at_5_std + value: 27.038595761090995 + - type: nauc_precision_at_1000_diff1 + value: 8.820420006228579 + - type: nauc_precision_at_1000_max + value: 27.589962032103767 + - type: nauc_precision_at_1000_std + value: 55.91804871658855 + - type: nauc_precision_at_100_diff1 + value: 1.861909084323091 + - type: nauc_precision_at_100_max + value: 24.679468330371577 + - type: nauc_precision_at_100_std + value: 46.971622410582256 + - type: nauc_precision_at_10_diff1 + value: 0.9473942721845824 + - type: nauc_precision_at_10_max + value: 27.373995617238855 + - type: nauc_precision_at_10_std + value: 33.8121206014962 + - type: nauc_precision_at_1_diff1 + value: 13.58643817493768 + - type: nauc_precision_at_1_max + value: 21.816447712518837 + - type: nauc_precision_at_1_std + value: 17.924499276989756 + - type: nauc_precision_at_20_diff1 + value: -0.9496032133443041 + - type: nauc_precision_at_20_max + value: 25.09536129203218 + - type: nauc_precision_at_20_std + value: 34.94477064476087 + - type: nauc_precision_at_3_diff1 + value: -1.6270834605287547 + - type: nauc_precision_at_3_max + value: 32.97922136068537 + - type: nauc_precision_at_3_std + value: 27.004090603821968 + - type: nauc_precision_at_5_diff1 + value: 3.152498000145442 + - type: nauc_precision_at_5_max + value: 28.41066104283325 + - type: nauc_precision_at_5_std + value: 28.703367027852533 + - type: nauc_recall_at_1000_diff1 + value: -6.851615969002735 + - type: nauc_recall_at_1000_max + value: 18.254946709759913 + - type: nauc_recall_at_1000_std + value: 44.660282645336395 + - type: nauc_recall_at_100_diff1 + value: -15.060165572281289 + - type: nauc_recall_at_100_max + value: 14.287315673289354 + - type: nauc_recall_at_100_std + value: 24.585719052074662 + - type: nauc_recall_at_10_diff1 + value: -0.5156087338678611 + - type: nauc_recall_at_10_max + value: 20.63199550876313 + - type: nauc_recall_at_10_std + value: 16.05339234192207 + - type: nauc_recall_at_1_diff1 + value: 2.163025554719988 + - type: nauc_recall_at_1_max + value: 4.173360349606284 + - type: nauc_recall_at_1_std + value: 3.8151957830662955 + - type: nauc_recall_at_20_diff1 + value: -7.154067602130297 + - type: nauc_recall_at_20_max + value: 15.224908904403495 + - type: nauc_recall_at_20_std + value: 12.626459111226785 + - type: nauc_recall_at_3_diff1 + value: -3.619649071909071 + - type: nauc_recall_at_3_max + value: 18.4012808687954 + - type: nauc_recall_at_3_std + value: 7.9348810473575115 + - type: nauc_recall_at_5_diff1 + value: 1.7480268902181528 + - type: nauc_recall_at_5_max + value: 18.308076072154737 + - type: nauc_recall_at_5_std + value: 8.189069818854295 + - type: ndcg_at_1 + value: 56.00000000000001 + - type: ndcg_at_10 + value: 43.236000000000004 + - type: ndcg_at_100 + value: 32.002 + - type: ndcg_at_1000 + value: 28.841 + - type: ndcg_at_20 + value: 41.939 + - type: ndcg_at_3 + value: 49.397000000000006 + - type: ndcg_at_5 + value: 45.989000000000004 + - type: precision_at_1 + value: 62.0 + - type: precision_at_10 + value: 45.2 + - type: precision_at_100 + value: 32.98 + - type: precision_at_1000 + value: 13.654 + - type: precision_at_20 + value: 44.6 + - type: precision_at_3 + value: 52.0 + - type: precision_at_5 + value: 48.0 + - type: recall_at_1 + value: 0.145 + - type: recall_at_10 + value: 1.095 + - type: recall_at_100 + value: 7.23 + - type: recall_at_1000 + value: 27.381 + - type: recall_at_20 + value: 2.137 + - type: recall_at_3 + value: 0.38 + - type: recall_at_5 + value: 0.598 + task: + type: Retrieval + - dataset: + config: default + name: MTEB Touche2020 (default) + revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f + split: test + type: mteb/touche2020 + metrics: + - type: main_score + value: 15.369 + - type: map_at_1 + value: 0.624 + - type: map_at_10 + value: 5.045999999999999 + - type: map_at_100 + value: 9.263 + - type: map_at_1000 + value: 10.685 + - type: map_at_20 + value: 6.802999999999999 + - type: map_at_3 + value: 2.096 + - type: map_at_5 + value: 3.0540000000000003 + - type: mrr_at_1 + value: 8.16326530612245 + - type: mrr_at_10 + value: 27.867670877874957 + - type: mrr_at_100 + value: 29.547080186435025 + - type: mrr_at_1000 + value: 29.547080186435025 + - type: mrr_at_20 + value: 29.159558218365945 + - type: mrr_at_3 + value: 23.809523809523807 + - type: mrr_at_5 + value: 26.462585034013603 + - type: nauc_map_at_1000_diff1 + value: 1.0582929159944574 + - type: nauc_map_at_1000_max + value: -27.972526856689594 + - type: nauc_map_at_1000_std + value: -13.966866642399777 + - type: nauc_map_at_100_diff1 + value: 1.6049379176721044 + - type: nauc_map_at_100_max + value: -27.93768261006475 + - type: nauc_map_at_100_std + value: -19.120569812088558 + - type: nauc_map_at_10_diff1 + value: -4.551284011489953 + - type: nauc_map_at_10_max + value: -36.348822221003736 + - type: nauc_map_at_10_std + value: -25.310440699574926 + - type: nauc_map_at_1_diff1 + value: -4.29133354793351 + - type: nauc_map_at_1_max + value: -33.683902237924606 + - type: nauc_map_at_1_std + value: -2.149083372866397 + - type: nauc_map_at_20_diff1 + value: -2.4993677210093415 + - type: nauc_map_at_20_max + value: -34.56781126360638 + - type: nauc_map_at_20_std + value: -27.270046086050996 + - type: nauc_map_at_3_diff1 + value: -8.863535339368541 + - type: nauc_map_at_3_max + value: -33.915423919314165 + - type: nauc_map_at_3_std + value: -20.931876097914717 + - type: nauc_map_at_5_diff1 + value: -6.112924399379883 + - type: nauc_map_at_5_max + value: -35.51745928601429 + - type: nauc_map_at_5_std + value: -26.71283272416072 + - type: nauc_mrr_at_1000_diff1 + value: -7.143214264658434 + - type: nauc_mrr_at_1000_max + value: -31.863746808733406 + - type: nauc_mrr_at_1000_std + value: -20.83112018798552 + - type: nauc_mrr_at_100_diff1 + value: -7.143214264658434 + - type: nauc_mrr_at_100_max + value: -31.863746808733406 + - type: nauc_mrr_at_100_std + value: -20.83112018798552 + - type: nauc_mrr_at_10_diff1 + value: -5.884787315885467 + - type: nauc_mrr_at_10_max + value: -30.635220886655258 + - type: nauc_mrr_at_10_std + value: -19.490662014520957 + - type: nauc_mrr_at_1_diff1 + value: -19.87621826159222 + - type: nauc_mrr_at_1_max + value: -29.000455091255144 + - type: nauc_mrr_at_1_std + value: -14.74961781962394 + - type: nauc_mrr_at_20_diff1 + value: -6.586786522775319 + - type: nauc_mrr_at_20_max + value: -31.762465562178672 + - type: nauc_mrr_at_20_std + value: -20.175801528907265 + - type: nauc_mrr_at_3_diff1 + value: -7.379742423129064 + - type: nauc_mrr_at_3_max + value: -29.391649545988567 + - type: nauc_mrr_at_3_std + value: -21.55556068358554 + - type: nauc_mrr_at_5_diff1 + value: -8.355170805869708 + - type: nauc_mrr_at_5_max + value: -31.372339495284457 + - type: nauc_mrr_at_5_std + value: -22.75382295756138 + - type: nauc_ndcg_at_1000_diff1 + value: 11.109947409844976 + - type: nauc_ndcg_at_1000_max + value: -23.497394105384846 + - type: nauc_ndcg_at_1000_std + value: 10.33864673546571 + - type: nauc_ndcg_at_100_diff1 + value: 10.309963888321153 + - type: nauc_ndcg_at_100_max + value: -29.80240939206779 + - type: nauc_ndcg_at_100_std + value: -12.225609266444376 + - type: nauc_ndcg_at_10_diff1 + value: 1.0618079883003713 + - type: nauc_ndcg_at_10_max + value: -30.20694258627494 + - type: nauc_ndcg_at_10_std + value: -20.899235269598666 + - type: nauc_ndcg_at_1_diff1 + value: -19.756295296761344 + - type: nauc_ndcg_at_1_max + value: -28.516727659678804 + - type: nauc_ndcg_at_1_std + value: -10.80797592567357 + - type: nauc_ndcg_at_20_diff1 + value: 5.732615761799287 + - type: nauc_ndcg_at_20_max + value: -34.92333737881057 + - type: nauc_ndcg_at_20_std + value: -26.461162552758942 + - type: nauc_ndcg_at_3_diff1 + value: -14.660143720588037 + - type: nauc_ndcg_at_3_max + value: -26.62064265146035 + - type: nauc_ndcg_at_3_std + value: -18.157487966395067 + - type: nauc_ndcg_at_5_diff1 + value: -7.731517563034629 + - type: nauc_ndcg_at_5_max + value: -27.00395580552045 + - type: nauc_ndcg_at_5_std + value: -24.039383478902586 + - type: nauc_precision_at_1000_diff1 + value: -4.030522868525787 + - type: nauc_precision_at_1000_max + value: 35.26062641499173 + - type: nauc_precision_at_1000_std + value: 56.17859102879932 + - type: nauc_precision_at_100_diff1 + value: 4.666878915081701 + - type: nauc_precision_at_100_max + value: -6.403637691779489 + - type: nauc_precision_at_100_std + value: 11.07070377101311 + - type: nauc_precision_at_10_diff1 + value: 0.05509998588231166 + - type: nauc_precision_at_10_max + value: -28.377648043763603 + - type: nauc_precision_at_10_std + value: -21.34157508908641 + - type: nauc_precision_at_1_diff1 + value: -19.87621826159222 + - type: nauc_precision_at_1_max + value: -29.000455091255144 + - type: nauc_precision_at_1_std + value: -14.74961781962394 + - type: nauc_precision_at_20_diff1 + value: 4.05836071238904 + - type: nauc_precision_at_20_max + value: -24.70626402704997 + - type: nauc_precision_at_20_std + value: -24.319709234669276 + - type: nauc_precision_at_3_diff1 + value: -14.934997211713059 + - type: nauc_precision_at_3_max + value: -25.535532316716708 + - type: nauc_precision_at_3_std + value: -22.977374631751008 + - type: nauc_precision_at_5_diff1 + value: -6.4331893193391405 + - type: nauc_precision_at_5_max + value: -24.72694882355003 + - type: nauc_precision_at_5_std + value: -27.54507044990821 + - type: nauc_recall_at_1000_diff1 + value: 12.952574652534027 + - type: nauc_recall_at_1000_max + value: -19.55020441237283 + - type: nauc_recall_at_1000_std + value: 44.97172549500198 + - type: nauc_recall_at_100_diff1 + value: 8.529360939874099 + - type: nauc_recall_at_100_max + value: -30.32863940847708 + - type: nauc_recall_at_100_std + value: -9.61948332450022 + - type: nauc_recall_at_10_diff1 + value: 2.4975692103190292 + - type: nauc_recall_at_10_max + value: -38.15178448204205 + - type: nauc_recall_at_10_std + value: -25.119879013303205 + - type: nauc_recall_at_1_diff1 + value: -4.29133354793351 + - type: nauc_recall_at_1_max + value: -33.683902237924606 + - type: nauc_recall_at_1_std + value: -2.149083372866397 + - type: nauc_recall_at_20_diff1 + value: 6.812191596907528 + - type: nauc_recall_at_20_max + value: -38.779604712399255 + - type: nauc_recall_at_20_std + value: -31.097428703581365 + - type: nauc_recall_at_3_diff1 + value: -5.858755837786076 + - type: nauc_recall_at_3_max + value: -36.38554330826264 + - type: nauc_recall_at_3_std + value: -27.587992095769547 + - type: nauc_recall_at_5_diff1 + value: -2.4757070734304856 + - type: nauc_recall_at_5_max + value: -38.80369033497341 + - type: nauc_recall_at_5_std + value: -32.378290142635706 + - type: ndcg_at_1 + value: 7.142999999999999 + - type: ndcg_at_10 + value: 15.369 + - type: ndcg_at_100 + value: 26.151999999999997 + - type: ndcg_at_1000 + value: 38.553 + - type: ndcg_at_20 + value: 17.325 + - type: ndcg_at_3 + value: 13.944999999999999 + - type: ndcg_at_5 + value: 14.565 + - type: precision_at_1 + value: 8.163 + - type: precision_at_10 + value: 16.735 + - type: precision_at_100 + value: 6.223999999999999 + - type: precision_at_1000 + value: 1.4200000000000002 + - type: precision_at_20 + value: 13.776 + - type: precision_at_3 + value: 17.687 + - type: precision_at_5 + value: 17.959 + - type: recall_at_1 + value: 0.624 + - type: recall_at_10 + value: 11.723 + - type: recall_at_100 + value: 38.072 + - type: recall_at_1000 + value: 76.011 + - type: recall_at_20 + value: 18.553 + - type: recall_at_3 + value: 3.7379999999999995 + - type: recall_at_5 + value: 6.2170000000000005 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ToxicConversationsClassification (default) + revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de + split: test + type: mteb/toxic_conversations_50k + metrics: + - type: accuracy + value: 67.724609375 + - type: ap + value: 12.118510465436508 + - type: ap_weighted + value: 12.118510465436508 + - type: f1 + value: 51.502156902277044 + - type: f1_weighted + value: 74.90031680337547 + - type: main_score + value: 67.724609375 + task: + type: Classification + - dataset: + config: default + name: MTEB TweetSentimentExtractionClassification (default) + revision: d604517c81ca91fe16a244d1248fc021f9ecee7a + split: test + type: mteb/tweet_sentiment_extraction + metrics: + - type: accuracy + value: 54.20486700622524 + - type: f1 + value: 54.430757063175975 + - type: f1_weighted + value: 53.770651858268025 + - type: main_score + value: 54.20486700622524 + task: + type: Classification + - dataset: + config: default + name: MTEB TwentyNewsgroupsClustering (default) + revision: 6125ec4e24fa026cec8a478383ee943acfbd5449 + split: test + type: mteb/twentynewsgroups-clustering + metrics: + - type: main_score + value: 31.416458059499856 + - type: v_measure + value: 31.416458059499856 + - type: v_measure_std + value: 1.7923550883286548 + task: + type: Clustering + - dataset: + config: default + name: MTEB TwitterSemEval2015 (default) + revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1 + split: test + type: mteb/twittersemeval2015-pairclassification + metrics: + - type: cosine_accuracy + value: 81.09912380044108 + - type: cosine_accuracy_threshold + value: 78.23829613291026 + - type: cosine_ap + value: 55.35060563890305 + - type: cosine_f1 + value: 53.80317025886646 + - type: cosine_f1_threshold + value: 66.90525865927196 + - type: cosine_precision + value: 47.37899176541474 + - type: cosine_recall + value: 62.24274406332454 + - type: dot_accuracy + value: 81.09912380044108 + - type: dot_accuracy_threshold + value: 78.23829756344699 + - type: dot_ap + value: 55.35060534827081 + - type: dot_f1 + value: 53.80317025886646 + - type: dot_f1_threshold + value: 66.90525431322651 + - type: dot_precision + value: 47.37899176541474 + - type: dot_recall + value: 62.24274406332454 + - type: euclidean_accuracy + value: 81.09912380044108 + - type: euclidean_accuracy_threshold + value: 65.97227331408025 + - type: euclidean_ap + value: 55.350603467922014 + - type: euclidean_f1 + value: 53.80317025886646 + - type: euclidean_f1_threshold + value: 81.35691587916897 + - type: euclidean_precision + value: 47.37899176541474 + - type: euclidean_recall + value: 62.24274406332454 + - type: main_score + value: 55.411799272092175 + - type: manhattan_accuracy + value: 81.25409787208679 + - type: manhattan_accuracy_threshold + value: 593.7671894840605 + - type: manhattan_ap + value: 55.411799272092175 + - type: manhattan_f1 + value: 53.751250416805604 + - type: manhattan_f1_threshold + value: 732.5377952336567 + - type: manhattan_precision + value: 46.43748799692721 + - type: manhattan_recall + value: 63.79947229551451 + - type: max_accuracy + value: 81.25409787208679 + - type: max_ap + value: 55.411799272092175 + - type: max_f1 + value: 53.80317025886646 + - type: max_precision + value: 47.37899176541474 + - type: max_recall + value: 63.79947229551451 + - type: similarity_accuracy + value: 81.09912380044108 + - type: similarity_accuracy_threshold + value: 78.23829613291026 + - type: similarity_ap + value: 55.35060563890305 + - type: similarity_f1 + value: 53.80317025886646 + - type: similarity_f1_threshold + value: 66.90525865927196 + - type: similarity_precision + value: 47.37899176541474 + - type: similarity_recall + value: 62.24274406332454 + task: + type: PairClassification + - dataset: + config: default + name: MTEB TwitterURLCorpus (default) + revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf + split: test + type: mteb/twitterurlcorpus-pairclassification + metrics: + - type: cosine_accuracy + value: 86.66123336049986 + - type: cosine_accuracy_threshold + value: 65.02320188247097 + - type: cosine_ap + value: 80.81687896238907 + - type: cosine_f1 + value: 73.12032134380135 + - type: cosine_f1_threshold + value: 58.973089149680625 + - type: cosine_precision + value: 69.54226574980899 + - type: cosine_recall + value: 77.08654142285187 + - type: dot_accuracy + value: 86.66123336049986 + - type: dot_accuracy_threshold + value: 65.02319875482263 + - type: dot_ap + value: 80.81688006755317 + - type: dot_f1 + value: 73.12032134380135 + - type: dot_f1_threshold + value: 58.97308824803296 + - type: dot_precision + value: 69.54226574980899 + - type: dot_recall + value: 77.08654142285187 + - type: euclidean_accuracy + value: 86.66123336049986 + - type: euclidean_accuracy_threshold + value: 83.63826448345795 + - type: euclidean_ap + value: 80.8168799506033 + - type: euclidean_f1 + value: 73.12032134380135 + - type: euclidean_f1_threshold + value: 90.58356387644642 + - type: euclidean_precision + value: 69.54226574980899 + - type: euclidean_recall + value: 77.08654142285187 + - type: main_score + value: 80.81688006755317 + - type: manhattan_accuracy + value: 86.5564481701401 + - type: manhattan_accuracy_threshold + value: 762.9837047696128 + - type: manhattan_ap + value: 80.69253277787045 + - type: manhattan_f1 + value: 73.04996608636671 + - type: manhattan_f1_threshold + value: 792.7533836003931 + - type: manhattan_precision + value: 71.5350553505535 + - type: manhattan_recall + value: 74.63042808746535 + - type: max_accuracy + value: 86.66123336049986 + - type: max_ap + value: 80.81688006755317 + - type: max_f1 + value: 73.12032134380135 + - type: max_precision + value: 71.5350553505535 + - type: max_recall + value: 77.08654142285187 + - type: similarity_accuracy + value: 86.66123336049986 + - type: similarity_accuracy_threshold + value: 65.02320188247097 + - type: similarity_ap + value: 80.81687896238907 + - type: similarity_f1 + value: 73.12032134380135 + - type: similarity_f1_threshold + value: 58.973089149680625 + - type: similarity_precision + value: 69.54226574980899 + - type: similarity_recall + value: 77.08654142285187 + task: + type: PairClassification +model_name: potion-base-4M +tags: +- embeddings +- static-embeddings +- mteb +--- + +# potion-base-4M Model Card + +This [Model2Vec](https://github.com/MinishLab/model2vec) model is pre-trained using [Tokenlearn](https://github.com/MinishLab/tokenlearn). It is a distilled version of the [baai/bge-base-en-v1.5](https://huggingface.co/baai/bge-base-en-v1.5) Sentence Transformer. It uses static embeddings, allowing text embeddings to be computed orders of magnitude faster on both GPU and CPU. It is designed for applications where computational resources are limited or where real-time performance is critical. + + + +## Installation + +Install model2vec using pip: +``` +pip install model2vec +``` + +## Usage +Load this model using the `from_pretrained` method: +```python +from model2vec import StaticModel + +# Load a pretrained Model2Vec model +model = StaticModel.from_pretrained("minishlab/potion-base-4M") + +# Compute text embeddings +embeddings = model.encode(["Example sentence"]) +``` + + +## How it works + +Model2vec creates a small, static model that outperforms other static embedding models by a large margin on all tasks on [MTEB](https://huggingface.co/spaces/mteb/leaderboard). This model is pre-trained using [Tokenlearn](https://github.com/MinishLab/tokenlearn). It's created using the following steps: +- Distillation: first, a model is distilled from a sentence transformer model using Model2Vec. +- Training data creation: the sentence transformer model is used to create training data by creating mean output embeddings on a large corpus. +- Training: the distilled model is trained on the training data using Tokenlearn. +- Post-training re-regularization: after training, the model is re-regularized by weighting the tokens based on their frequency, applying PCA, and finally applying [SIF weighting](https://openreview.net/pdf?id=SyK00v5xx). + +The results for this model can be found on the [Model2Vec results page](https://github.com/MinishLab/model2vec/blob/main/results/README.md). + +## Additional Resources + +- [All Model2Vec models on the hub](https://huggingface.co/models?library=model2vec) +- [Model2Vec Repo](https://github.com/MinishLab/model2vec) +- [Tokenlearn repo](https://github.com/MinishLab/tokenlearn) +- [Model2Vec Results](https://github.com/MinishLab/model2vec/blob/main/results/README.md) +- [Model2Vec Tutorials](https://github.com/MinishLab/model2vec/tree/main/tutorials) + +## Library Authors + +Model2Vec was developed by the [Minish Lab](https://github.com/MinishLab) team consisting of [Stephan Tulkens](https://github.com/stephantul) and [Thomas van Dongen](https://github.com/Pringled). + +## Citation + +Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work. +``` +@software{minishlab2024model2vec, + authors = {Stephan Tulkens, Thomas van Dongen}, + title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model}, + year = {2024}, + url = {https://github.com/MinishLab/model2vec}, +} +``` \ No newline at end of file