spacemanidol commited on
Commit
639bf59
1 Parent(s): 3b0645c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3017 -1
README.md CHANGED
@@ -2799,4 +2799,3020 @@ model-index:
2799
  metrics:
2800
  - type: v_measure
2801
  value: 79.58576208710117
2802
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2799
  metrics:
2800
  - type: v_measure
2801
  value: 79.58576208710117
2802
+ ---
2803
+ ---
2804
+ tags:
2805
+ - mteb
2806
+ - arctic
2807
+ - arctic-embed
2808
+ model-index:
2809
+ - name: base
2810
+ results:
2811
+ - task:
2812
+ type: Classification
2813
+ dataset:
2814
+ type: mteb/amazon_counterfactual
2815
+ name: MTEB AmazonCounterfactualClassification (en)
2816
+ config: en
2817
+ split: test
2818
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
2819
+ metrics:
2820
+ - type: accuracy
2821
+ value: 76.80597014925374
2822
+ - type: ap
2823
+ value: 39.31198155789558
2824
+ - type: f1
2825
+ value: 70.48198448222148
2826
+ - task:
2827
+ type: Classification
2828
+ dataset:
2829
+ type: mteb/amazon_polarity
2830
+ name: MTEB AmazonPolarityClassification
2831
+ config: default
2832
+ split: test
2833
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
2834
+ metrics:
2835
+ - type: accuracy
2836
+ value: 82.831525
2837
+ - type: ap
2838
+ value: 77.4474050181638
2839
+ - type: f1
2840
+ value: 82.77204845110204
2841
+ - task:
2842
+ type: Classification
2843
+ dataset:
2844
+ type: mteb/amazon_reviews_multi
2845
+ name: MTEB AmazonReviewsClassification (en)
2846
+ config: en
2847
+ split: test
2848
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
2849
+ metrics:
2850
+ - type: accuracy
2851
+ value: 38.93000000000001
2852
+ - type: f1
2853
+ value: 37.98013371053459
2854
+ - task:
2855
+ type: Retrieval
2856
+ dataset:
2857
+ type: mteb/arguana
2858
+ name: MTEB ArguAna
2859
+ config: default
2860
+ split: test
2861
+ revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
2862
+ metrics:
2863
+ - type: map_at_1
2864
+ value: 31.223
2865
+ - type: map_at_10
2866
+ value: 47.43
2867
+ - type: map_at_100
2868
+ value: 48.208
2869
+ - type: map_at_1000
2870
+ value: 48.211
2871
+ - type: map_at_3
2872
+ value: 42.579
2873
+ - type: map_at_5
2874
+ value: 45.263999999999996
2875
+ - type: mrr_at_1
2876
+ value: 31.65
2877
+ - type: mrr_at_10
2878
+ value: 47.573
2879
+ - type: mrr_at_100
2880
+ value: 48.359
2881
+ - type: mrr_at_1000
2882
+ value: 48.362
2883
+ - type: mrr_at_3
2884
+ value: 42.734
2885
+ - type: mrr_at_5
2886
+ value: 45.415
2887
+ - type: ndcg_at_1
2888
+ value: 31.223
2889
+ - type: ndcg_at_10
2890
+ value: 56.436
2891
+ - type: ndcg_at_100
2892
+ value: 59.657000000000004
2893
+ - type: ndcg_at_1000
2894
+ value: 59.731
2895
+ - type: ndcg_at_3
2896
+ value: 46.327
2897
+ - type: ndcg_at_5
2898
+ value: 51.178000000000004
2899
+ - type: precision_at_1
2900
+ value: 31.223
2901
+ - type: precision_at_10
2902
+ value: 8.527999999999999
2903
+ - type: precision_at_100
2904
+ value: 0.991
2905
+ - type: precision_at_1000
2906
+ value: 0.1
2907
+ - type: precision_at_3
2908
+ value: 19.061
2909
+ - type: precision_at_5
2910
+ value: 13.797999999999998
2911
+ - type: recall_at_1
2912
+ value: 31.223
2913
+ - type: recall_at_10
2914
+ value: 85.277
2915
+ - type: recall_at_100
2916
+ value: 99.075
2917
+ - type: recall_at_1000
2918
+ value: 99.644
2919
+ - type: recall_at_3
2920
+ value: 57.18299999999999
2921
+ - type: recall_at_5
2922
+ value: 68.99
2923
+ - task:
2924
+ type: Clustering
2925
+ dataset:
2926
+ type: mteb/arxiv-clustering-p2p
2927
+ name: MTEB ArxivClusteringP2P
2928
+ config: default
2929
+ split: test
2930
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
2931
+ metrics:
2932
+ - type: v_measure
2933
+ value: 47.23625429411296
2934
+ - task:
2935
+ type: Clustering
2936
+ dataset:
2937
+ type: mteb/arxiv-clustering-s2s
2938
+ name: MTEB ArxivClusteringS2S
2939
+ config: default
2940
+ split: test
2941
+ revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
2942
+ metrics:
2943
+ - type: v_measure
2944
+ value: 37.433880471403654
2945
+ - task:
2946
+ type: Reranking
2947
+ dataset:
2948
+ type: mteb/askubuntudupquestions-reranking
2949
+ name: MTEB AskUbuntuDupQuestions
2950
+ config: default
2951
+ split: test
2952
+ revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
2953
+ metrics:
2954
+ - type: map
2955
+ value: 60.53175025582013
2956
+ - type: mrr
2957
+ value: 74.51160796728664
2958
+ - task:
2959
+ type: STS
2960
+ dataset:
2961
+ type: mteb/biosses-sts
2962
+ name: MTEB BIOSSES
2963
+ config: default
2964
+ split: test
2965
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
2966
+ metrics:
2967
+ - type: cos_sim_pearson
2968
+ value: 88.93746103286769
2969
+ - type: cos_sim_spearman
2970
+ value: 86.62245567912619
2971
+ - type: euclidean_pearson
2972
+ value: 87.154173907501
2973
+ - type: euclidean_spearman
2974
+ value: 86.62245567912619
2975
+ - type: manhattan_pearson
2976
+ value: 87.17682026633462
2977
+ - type: manhattan_spearman
2978
+ value: 86.74775973908348
2979
+ - task:
2980
+ type: Classification
2981
+ dataset:
2982
+ type: mteb/banking77
2983
+ name: MTEB Banking77Classification
2984
+ config: default
2985
+ split: test
2986
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
2987
+ metrics:
2988
+ - type: accuracy
2989
+ value: 80.33766233766232
2990
+ - type: f1
2991
+ value: 79.64931422442245
2992
+ - task:
2993
+ type: Clustering
2994
+ dataset:
2995
+ type: jinaai/big-patent-clustering
2996
+ name: MTEB BigPatentClustering
2997
+ config: default
2998
+ split: test
2999
+ revision: 62d5330920bca426ce9d3c76ea914f15fc83e891
3000
+ metrics:
3001
+ - type: v_measure
3002
+ value: 19.116028913890613
3003
+ - task:
3004
+ type: Clustering
3005
+ dataset:
3006
+ type: mteb/biorxiv-clustering-p2p
3007
+ name: MTEB BiorxivClusteringP2P
3008
+ config: default
3009
+ split: test
3010
+ revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
3011
+ metrics:
3012
+ - type: v_measure
3013
+ value: 36.966921852810174
3014
+ - task:
3015
+ type: Clustering
3016
+ dataset:
3017
+ type: mteb/biorxiv-clustering-s2s
3018
+ name: MTEB BiorxivClusteringS2S
3019
+ config: default
3020
+ split: test
3021
+ revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
3022
+ metrics:
3023
+ - type: v_measure
3024
+ value: 31.98019698537654
3025
+ - task:
3026
+ type: Retrieval
3027
+ dataset:
3028
+ type: mteb/cqadupstack-android
3029
+ name: MTEB CQADupstackAndroidRetrieval
3030
+ config: default
3031
+ split: test
3032
+ revision: f46a197baaae43b4f621051089b82a364682dfeb
3033
+ metrics:
3034
+ - type: map_at_1
3035
+ value: 34.079
3036
+ - type: map_at_10
3037
+ value: 46.35
3038
+ - type: map_at_100
3039
+ value: 47.785
3040
+ - type: map_at_1000
3041
+ value: 47.903
3042
+ - type: map_at_3
3043
+ value: 42.620999999999995
3044
+ - type: map_at_5
3045
+ value: 44.765
3046
+ - type: mrr_at_1
3047
+ value: 41.345
3048
+ - type: mrr_at_10
3049
+ value: 52.032000000000004
3050
+ - type: mrr_at_100
3051
+ value: 52.690000000000005
3052
+ - type: mrr_at_1000
3053
+ value: 52.727999999999994
3054
+ - type: mrr_at_3
3055
+ value: 49.428
3056
+ - type: mrr_at_5
3057
+ value: 51.093999999999994
3058
+ - type: ndcg_at_1
3059
+ value: 41.345
3060
+ - type: ndcg_at_10
3061
+ value: 53.027
3062
+ - type: ndcg_at_100
3063
+ value: 57.962
3064
+ - type: ndcg_at_1000
3065
+ value: 59.611999999999995
3066
+ - type: ndcg_at_3
3067
+ value: 47.687000000000005
3068
+ - type: ndcg_at_5
3069
+ value: 50.367
3070
+ - type: precision_at_1
3071
+ value: 41.345
3072
+ - type: precision_at_10
3073
+ value: 10.157
3074
+ - type: precision_at_100
3075
+ value: 1.567
3076
+ - type: precision_at_1000
3077
+ value: 0.199
3078
+ - type: precision_at_3
3079
+ value: 23.081
3080
+ - type: precision_at_5
3081
+ value: 16.738
3082
+ - type: recall_at_1
3083
+ value: 34.079
3084
+ - type: recall_at_10
3085
+ value: 65.93900000000001
3086
+ - type: recall_at_100
3087
+ value: 86.42699999999999
3088
+ - type: recall_at_1000
3089
+ value: 96.61
3090
+ - type: recall_at_3
3091
+ value: 50.56699999999999
3092
+ - type: recall_at_5
3093
+ value: 57.82000000000001
3094
+ - task:
3095
+ type: Retrieval
3096
+ dataset:
3097
+ type: mteb/cqadupstack-english
3098
+ name: MTEB CQADupstackEnglishRetrieval
3099
+ config: default
3100
+ split: test
3101
+ revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
3102
+ metrics:
3103
+ - type: map_at_1
3104
+ value: 33.289
3105
+ - type: map_at_10
3106
+ value: 43.681
3107
+ - type: map_at_100
3108
+ value: 45.056000000000004
3109
+ - type: map_at_1000
3110
+ value: 45.171
3111
+ - type: map_at_3
3112
+ value: 40.702
3113
+ - type: map_at_5
3114
+ value: 42.292
3115
+ - type: mrr_at_1
3116
+ value: 41.146
3117
+ - type: mrr_at_10
3118
+ value: 49.604
3119
+ - type: mrr_at_100
3120
+ value: 50.28399999999999
3121
+ - type: mrr_at_1000
3122
+ value: 50.322
3123
+ - type: mrr_at_3
3124
+ value: 47.611
3125
+ - type: mrr_at_5
3126
+ value: 48.717
3127
+ - type: ndcg_at_1
3128
+ value: 41.146
3129
+ - type: ndcg_at_10
3130
+ value: 49.43
3131
+ - type: ndcg_at_100
3132
+ value: 54.01899999999999
3133
+ - type: ndcg_at_1000
3134
+ value: 55.803000000000004
3135
+ - type: ndcg_at_3
3136
+ value: 45.503
3137
+ - type: ndcg_at_5
3138
+ value: 47.198
3139
+ - type: precision_at_1
3140
+ value: 41.146
3141
+ - type: precision_at_10
3142
+ value: 9.268
3143
+ - type: precision_at_100
3144
+ value: 1.4749999999999999
3145
+ - type: precision_at_1000
3146
+ value: 0.19
3147
+ - type: precision_at_3
3148
+ value: 21.932
3149
+ - type: precision_at_5
3150
+ value: 15.389
3151
+ - type: recall_at_1
3152
+ value: 33.289
3153
+ - type: recall_at_10
3154
+ value: 59.209999999999994
3155
+ - type: recall_at_100
3156
+ value: 78.676
3157
+ - type: recall_at_1000
3158
+ value: 89.84100000000001
3159
+ - type: recall_at_3
3160
+ value: 47.351
3161
+ - type: recall_at_5
3162
+ value: 52.178999999999995
3163
+ - task:
3164
+ type: Retrieval
3165
+ dataset:
3166
+ type: mteb/cqadupstack-gaming
3167
+ name: MTEB CQADupstackGamingRetrieval
3168
+ config: default
3169
+ split: test
3170
+ revision: 4885aa143210c98657558c04aaf3dc47cfb54340
3171
+ metrics:
3172
+ - type: map_at_1
3173
+ value: 44.483
3174
+ - type: map_at_10
3175
+ value: 56.862
3176
+ - type: map_at_100
3177
+ value: 57.901
3178
+ - type: map_at_1000
3179
+ value: 57.948
3180
+ - type: map_at_3
3181
+ value: 53.737
3182
+ - type: map_at_5
3183
+ value: 55.64
3184
+ - type: mrr_at_1
3185
+ value: 50.658
3186
+ - type: mrr_at_10
3187
+ value: 60.281
3188
+ - type: mrr_at_100
3189
+ value: 60.946
3190
+ - type: mrr_at_1000
3191
+ value: 60.967000000000006
3192
+ - type: mrr_at_3
3193
+ value: 58.192
3194
+ - type: mrr_at_5
3195
+ value: 59.531
3196
+ - type: ndcg_at_1
3197
+ value: 50.658
3198
+ - type: ndcg_at_10
3199
+ value: 62.339
3200
+ - type: ndcg_at_100
3201
+ value: 66.28399999999999
3202
+ - type: ndcg_at_1000
3203
+ value: 67.166
3204
+ - type: ndcg_at_3
3205
+ value: 57.458
3206
+ - type: ndcg_at_5
3207
+ value: 60.112
3208
+ - type: precision_at_1
3209
+ value: 50.658
3210
+ - type: precision_at_10
3211
+ value: 9.762
3212
+ - type: precision_at_100
3213
+ value: 1.26
3214
+ - type: precision_at_1000
3215
+ value: 0.13799999999999998
3216
+ - type: precision_at_3
3217
+ value: 25.329
3218
+ - type: precision_at_5
3219
+ value: 17.254
3220
+ - type: recall_at_1
3221
+ value: 44.483
3222
+ - type: recall_at_10
3223
+ value: 74.819
3224
+ - type: recall_at_100
3225
+ value: 91.702
3226
+ - type: recall_at_1000
3227
+ value: 97.84
3228
+ - type: recall_at_3
3229
+ value: 62.13999999999999
3230
+ - type: recall_at_5
3231
+ value: 68.569
3232
+ - task:
3233
+ type: Retrieval
3234
+ dataset:
3235
+ type: mteb/cqadupstack-gis
3236
+ name: MTEB CQADupstackGisRetrieval
3237
+ config: default
3238
+ split: test
3239
+ revision: 5003b3064772da1887988e05400cf3806fe491f2
3240
+ metrics:
3241
+ - type: map_at_1
3242
+ value: 26.489
3243
+ - type: map_at_10
3244
+ value: 37.004999999999995
3245
+ - type: map_at_100
3246
+ value: 38.001000000000005
3247
+ - type: map_at_1000
3248
+ value: 38.085
3249
+ - type: map_at_3
3250
+ value: 34.239999999999995
3251
+ - type: map_at_5
3252
+ value: 35.934
3253
+ - type: mrr_at_1
3254
+ value: 28.362
3255
+ - type: mrr_at_10
3256
+ value: 38.807
3257
+ - type: mrr_at_100
3258
+ value: 39.671
3259
+ - type: mrr_at_1000
3260
+ value: 39.736
3261
+ - type: mrr_at_3
3262
+ value: 36.29
3263
+ - type: mrr_at_5
3264
+ value: 37.906
3265
+ - type: ndcg_at_1
3266
+ value: 28.362
3267
+ - type: ndcg_at_10
3268
+ value: 42.510999999999996
3269
+ - type: ndcg_at_100
3270
+ value: 47.226
3271
+ - type: ndcg_at_1000
3272
+ value: 49.226
3273
+ - type: ndcg_at_3
3274
+ value: 37.295
3275
+ - type: ndcg_at_5
3276
+ value: 40.165
3277
+ - type: precision_at_1
3278
+ value: 28.362
3279
+ - type: precision_at_10
3280
+ value: 6.633
3281
+ - type: precision_at_100
3282
+ value: 0.9490000000000001
3283
+ - type: precision_at_1000
3284
+ value: 0.11499999999999999
3285
+ - type: precision_at_3
3286
+ value: 16.234
3287
+ - type: precision_at_5
3288
+ value: 11.434999999999999
3289
+ - type: recall_at_1
3290
+ value: 26.489
3291
+ - type: recall_at_10
3292
+ value: 57.457
3293
+ - type: recall_at_100
3294
+ value: 78.712
3295
+ - type: recall_at_1000
3296
+ value: 93.565
3297
+ - type: recall_at_3
3298
+ value: 43.748
3299
+ - type: recall_at_5
3300
+ value: 50.589
3301
+ - task:
3302
+ type: Retrieval
3303
+ dataset:
3304
+ type: mteb/cqadupstack-mathematica
3305
+ name: MTEB CQADupstackMathematicaRetrieval
3306
+ config: default
3307
+ split: test
3308
+ revision: 90fceea13679c63fe563ded68f3b6f06e50061de
3309
+ metrics:
3310
+ - type: map_at_1
3311
+ value: 12.418999999999999
3312
+ - type: map_at_10
3313
+ value: 22.866
3314
+ - type: map_at_100
3315
+ value: 24.365000000000002
3316
+ - type: map_at_1000
3317
+ value: 24.479
3318
+ - type: map_at_3
3319
+ value: 19.965
3320
+ - type: map_at_5
3321
+ value: 21.684
3322
+ - type: mrr_at_1
3323
+ value: 14.677000000000001
3324
+ - type: mrr_at_10
3325
+ value: 26.316
3326
+ - type: mrr_at_100
3327
+ value: 27.514
3328
+ - type: mrr_at_1000
3329
+ value: 27.57
3330
+ - type: mrr_at_3
3331
+ value: 23.3
3332
+ - type: mrr_at_5
3333
+ value: 25.191000000000003
3334
+ - type: ndcg_at_1
3335
+ value: 14.677000000000001
3336
+ - type: ndcg_at_10
3337
+ value: 28.875
3338
+ - type: ndcg_at_100
3339
+ value: 35.607
3340
+ - type: ndcg_at_1000
3341
+ value: 38.237
3342
+ - type: ndcg_at_3
3343
+ value: 23.284
3344
+ - type: ndcg_at_5
3345
+ value: 26.226
3346
+ - type: precision_at_1
3347
+ value: 14.677000000000001
3348
+ - type: precision_at_10
3349
+ value: 5.771
3350
+ - type: precision_at_100
3351
+ value: 1.058
3352
+ - type: precision_at_1000
3353
+ value: 0.14200000000000002
3354
+ - type: precision_at_3
3355
+ value: 11.940000000000001
3356
+ - type: precision_at_5
3357
+ value: 9.229
3358
+ - type: recall_at_1
3359
+ value: 12.418999999999999
3360
+ - type: recall_at_10
3361
+ value: 43.333
3362
+ - type: recall_at_100
3363
+ value: 71.942
3364
+ - type: recall_at_1000
3365
+ value: 90.67399999999999
3366
+ - type: recall_at_3
3367
+ value: 28.787000000000003
3368
+ - type: recall_at_5
3369
+ value: 35.638
3370
+ - task:
3371
+ type: Retrieval
3372
+ dataset:
3373
+ type: mteb/cqadupstack-physics
3374
+ name: MTEB CQADupstackPhysicsRetrieval
3375
+ config: default
3376
+ split: test
3377
+ revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
3378
+ metrics:
3379
+ - type: map_at_1
3380
+ value: 31.686999999999998
3381
+ - type: map_at_10
3382
+ value: 42.331
3383
+ - type: map_at_100
3384
+ value: 43.655
3385
+ - type: map_at_1000
3386
+ value: 43.771
3387
+ - type: map_at_3
3388
+ value: 38.944
3389
+ - type: map_at_5
3390
+ value: 40.991
3391
+ - type: mrr_at_1
3392
+ value: 37.921
3393
+ - type: mrr_at_10
3394
+ value: 47.534
3395
+ - type: mrr_at_100
3396
+ value: 48.362
3397
+ - type: mrr_at_1000
3398
+ value: 48.405
3399
+ - type: mrr_at_3
3400
+ value: 44.995000000000005
3401
+ - type: mrr_at_5
3402
+ value: 46.617
3403
+ - type: ndcg_at_1
3404
+ value: 37.921
3405
+ - type: ndcg_at_10
3406
+ value: 48.236000000000004
3407
+ - type: ndcg_at_100
3408
+ value: 53.705000000000005
3409
+ - type: ndcg_at_1000
3410
+ value: 55.596000000000004
3411
+ - type: ndcg_at_3
3412
+ value: 43.11
3413
+ - type: ndcg_at_5
3414
+ value: 45.862
3415
+ - type: precision_at_1
3416
+ value: 37.921
3417
+ - type: precision_at_10
3418
+ value: 8.643
3419
+ - type: precision_at_100
3420
+ value: 1.336
3421
+ - type: precision_at_1000
3422
+ value: 0.166
3423
+ - type: precision_at_3
3424
+ value: 20.308
3425
+ - type: precision_at_5
3426
+ value: 14.514
3427
+ - type: recall_at_1
3428
+ value: 31.686999999999998
3429
+ - type: recall_at_10
3430
+ value: 60.126999999999995
3431
+ - type: recall_at_100
3432
+ value: 83.10600000000001
3433
+ - type: recall_at_1000
3434
+ value: 95.15
3435
+ - type: recall_at_3
3436
+ value: 46.098
3437
+ - type: recall_at_5
3438
+ value: 53.179
3439
+ - task:
3440
+ type: Retrieval
3441
+ dataset:
3442
+ type: mteb/cqadupstack-programmers
3443
+ name: MTEB CQADupstackProgrammersRetrieval
3444
+ config: default
3445
+ split: test
3446
+ revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
3447
+ metrics:
3448
+ - type: map_at_1
3449
+ value: 28.686
3450
+ - type: map_at_10
3451
+ value: 39.146
3452
+ - type: map_at_100
3453
+ value: 40.543
3454
+ - type: map_at_1000
3455
+ value: 40.644999999999996
3456
+ - type: map_at_3
3457
+ value: 36.195
3458
+ - type: map_at_5
3459
+ value: 37.919000000000004
3460
+ - type: mrr_at_1
3461
+ value: 35.160000000000004
3462
+ - type: mrr_at_10
3463
+ value: 44.711
3464
+ - type: mrr_at_100
3465
+ value: 45.609
3466
+ - type: mrr_at_1000
3467
+ value: 45.655
3468
+ - type: mrr_at_3
3469
+ value: 42.409
3470
+ - type: mrr_at_5
3471
+ value: 43.779
3472
+ - type: ndcg_at_1
3473
+ value: 35.160000000000004
3474
+ - type: ndcg_at_10
3475
+ value: 44.977000000000004
3476
+ - type: ndcg_at_100
3477
+ value: 50.663000000000004
3478
+ - type: ndcg_at_1000
3479
+ value: 52.794
3480
+ - type: ndcg_at_3
3481
+ value: 40.532000000000004
3482
+ - type: ndcg_at_5
3483
+ value: 42.641
3484
+ - type: precision_at_1
3485
+ value: 35.160000000000004
3486
+ - type: precision_at_10
3487
+ value: 8.014000000000001
3488
+ - type: precision_at_100
3489
+ value: 1.269
3490
+ - type: precision_at_1000
3491
+ value: 0.163
3492
+ - type: precision_at_3
3493
+ value: 19.444
3494
+ - type: precision_at_5
3495
+ value: 13.653
3496
+ - type: recall_at_1
3497
+ value: 28.686
3498
+ - type: recall_at_10
3499
+ value: 56.801
3500
+ - type: recall_at_100
3501
+ value: 80.559
3502
+ - type: recall_at_1000
3503
+ value: 95.052
3504
+ - type: recall_at_3
3505
+ value: 43.675999999999995
3506
+ - type: recall_at_5
3507
+ value: 49.703
3508
+ - task:
3509
+ type: Retrieval
3510
+ dataset:
3511
+ type: mteb/cqadupstack
3512
+ name: MTEB CQADupstackRetrieval
3513
+ config: default
3514
+ split: test
3515
+ revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
3516
+ metrics:
3517
+ - type: map_at_1
3518
+ value: 28.173833333333338
3519
+ - type: map_at_10
3520
+ value: 38.202083333333334
3521
+ - type: map_at_100
3522
+ value: 39.47475
3523
+ - type: map_at_1000
3524
+ value: 39.586499999999994
3525
+ - type: map_at_3
3526
+ value: 35.17308333333334
3527
+ - type: map_at_5
3528
+ value: 36.914
3529
+ - type: mrr_at_1
3530
+ value: 32.92958333333333
3531
+ - type: mrr_at_10
3532
+ value: 42.16758333333333
3533
+ - type: mrr_at_100
3534
+ value: 43.04108333333333
3535
+ - type: mrr_at_1000
3536
+ value: 43.092499999999994
3537
+ - type: mrr_at_3
3538
+ value: 39.69166666666666
3539
+ - type: mrr_at_5
3540
+ value: 41.19458333333333
3541
+ - type: ndcg_at_1
3542
+ value: 32.92958333333333
3543
+ - type: ndcg_at_10
3544
+ value: 43.80583333333333
3545
+ - type: ndcg_at_100
3546
+ value: 49.060916666666664
3547
+ - type: ndcg_at_1000
3548
+ value: 51.127250000000004
3549
+ - type: ndcg_at_3
3550
+ value: 38.80383333333333
3551
+ - type: ndcg_at_5
3552
+ value: 41.29658333333333
3553
+ - type: precision_at_1
3554
+ value: 32.92958333333333
3555
+ - type: precision_at_10
3556
+ value: 7.655666666666666
3557
+ - type: precision_at_100
3558
+ value: 1.2094166666666668
3559
+ - type: precision_at_1000
3560
+ value: 0.15750000000000003
3561
+ - type: precision_at_3
3562
+ value: 17.87975
3563
+ - type: precision_at_5
3564
+ value: 12.741833333333332
3565
+ - type: recall_at_1
3566
+ value: 28.173833333333338
3567
+ - type: recall_at_10
3568
+ value: 56.219249999999995
3569
+ - type: recall_at_100
3570
+ value: 79.01416666666665
3571
+ - type: recall_at_1000
3572
+ value: 93.13425000000001
3573
+ - type: recall_at_3
3574
+ value: 42.39241666666667
3575
+ - type: recall_at_5
3576
+ value: 48.764833333333335
3577
+ - task:
3578
+ type: Retrieval
3579
+ dataset:
3580
+ type: mteb/cqadupstack-stats
3581
+ name: MTEB CQADupstackStatsRetrieval
3582
+ config: default
3583
+ split: test
3584
+ revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
3585
+ metrics:
3586
+ - type: map_at_1
3587
+ value: 25.625999999999998
3588
+ - type: map_at_10
3589
+ value: 32.808
3590
+ - type: map_at_100
3591
+ value: 33.951
3592
+ - type: map_at_1000
3593
+ value: 34.052
3594
+ - type: map_at_3
3595
+ value: 30.536
3596
+ - type: map_at_5
3597
+ value: 31.77
3598
+ - type: mrr_at_1
3599
+ value: 28.374
3600
+ - type: mrr_at_10
3601
+ value: 35.527
3602
+ - type: mrr_at_100
3603
+ value: 36.451
3604
+ - type: mrr_at_1000
3605
+ value: 36.522
3606
+ - type: mrr_at_3
3607
+ value: 33.410000000000004
3608
+ - type: mrr_at_5
3609
+ value: 34.537
3610
+ - type: ndcg_at_1
3611
+ value: 28.374
3612
+ - type: ndcg_at_10
3613
+ value: 37.172
3614
+ - type: ndcg_at_100
3615
+ value: 42.474000000000004
3616
+ - type: ndcg_at_1000
3617
+ value: 44.853
3618
+ - type: ndcg_at_3
3619
+ value: 32.931
3620
+ - type: ndcg_at_5
3621
+ value: 34.882999999999996
3622
+ - type: precision_at_1
3623
+ value: 28.374
3624
+ - type: precision_at_10
3625
+ value: 5.813
3626
+ - type: precision_at_100
3627
+ value: 0.928
3628
+ - type: precision_at_1000
3629
+ value: 0.121
3630
+ - type: precision_at_3
3631
+ value: 14.008000000000001
3632
+ - type: precision_at_5
3633
+ value: 9.754999999999999
3634
+ - type: recall_at_1
3635
+ value: 25.625999999999998
3636
+ - type: recall_at_10
3637
+ value: 47.812
3638
+ - type: recall_at_100
3639
+ value: 71.61800000000001
3640
+ - type: recall_at_1000
3641
+ value: 88.881
3642
+ - type: recall_at_3
3643
+ value: 35.876999999999995
3644
+ - type: recall_at_5
3645
+ value: 40.839
3646
+ - task:
3647
+ type: Retrieval
3648
+ dataset:
3649
+ type: mteb/cqadupstack-tex
3650
+ name: MTEB CQADupstackTexRetrieval
3651
+ config: default
3652
+ split: test
3653
+ revision: 46989137a86843e03a6195de44b09deda022eec7
3654
+ metrics:
3655
+ - type: map_at_1
3656
+ value: 18.233
3657
+ - type: map_at_10
3658
+ value: 26.375999999999998
3659
+ - type: map_at_100
3660
+ value: 27.575
3661
+ - type: map_at_1000
3662
+ value: 27.706999999999997
3663
+ - type: map_at_3
3664
+ value: 23.619
3665
+ - type: map_at_5
3666
+ value: 25.217
3667
+ - type: mrr_at_1
3668
+ value: 22.023
3669
+ - type: mrr_at_10
3670
+ value: 30.122
3671
+ - type: mrr_at_100
3672
+ value: 31.083
3673
+ - type: mrr_at_1000
3674
+ value: 31.163999999999998
3675
+ - type: mrr_at_3
3676
+ value: 27.541
3677
+ - type: mrr_at_5
3678
+ value: 29.061999999999998
3679
+ - type: ndcg_at_1
3680
+ value: 22.023
3681
+ - type: ndcg_at_10
3682
+ value: 31.476
3683
+ - type: ndcg_at_100
3684
+ value: 37.114000000000004
3685
+ - type: ndcg_at_1000
3686
+ value: 39.981
3687
+ - type: ndcg_at_3
3688
+ value: 26.538
3689
+ - type: ndcg_at_5
3690
+ value: 29.016
3691
+ - type: precision_at_1
3692
+ value: 22.023
3693
+ - type: precision_at_10
3694
+ value: 5.819
3695
+ - type: precision_at_100
3696
+ value: 1.018
3697
+ - type: precision_at_1000
3698
+ value: 0.14300000000000002
3699
+ - type: precision_at_3
3700
+ value: 12.583
3701
+ - type: precision_at_5
3702
+ value: 9.36
3703
+ - type: recall_at_1
3704
+ value: 18.233
3705
+ - type: recall_at_10
3706
+ value: 43.029
3707
+ - type: recall_at_100
3708
+ value: 68.253
3709
+ - type: recall_at_1000
3710
+ value: 88.319
3711
+ - type: recall_at_3
3712
+ value: 29.541
3713
+ - type: recall_at_5
3714
+ value: 35.783
3715
+ - task:
3716
+ type: Retrieval
3717
+ dataset:
3718
+ type: mteb/cqadupstack-unix
3719
+ name: MTEB CQADupstackUnixRetrieval
3720
+ config: default
3721
+ split: test
3722
+ revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
3723
+ metrics:
3724
+ - type: map_at_1
3725
+ value: 28.923
3726
+ - type: map_at_10
3727
+ value: 39.231
3728
+ - type: map_at_100
3729
+ value: 40.483000000000004
3730
+ - type: map_at_1000
3731
+ value: 40.575
3732
+ - type: map_at_3
3733
+ value: 35.94
3734
+ - type: map_at_5
3735
+ value: 37.683
3736
+ - type: mrr_at_1
3737
+ value: 33.955
3738
+ - type: mrr_at_10
3739
+ value: 43.163000000000004
3740
+ - type: mrr_at_100
3741
+ value: 44.054
3742
+ - type: mrr_at_1000
3743
+ value: 44.099
3744
+ - type: mrr_at_3
3745
+ value: 40.361000000000004
3746
+ - type: mrr_at_5
3747
+ value: 41.905
3748
+ - type: ndcg_at_1
3749
+ value: 33.955
3750
+ - type: ndcg_at_10
3751
+ value: 45.068000000000005
3752
+ - type: ndcg_at_100
3753
+ value: 50.470000000000006
3754
+ - type: ndcg_at_1000
3755
+ value: 52.349000000000004
3756
+ - type: ndcg_at_3
3757
+ value: 39.298
3758
+ - type: ndcg_at_5
3759
+ value: 41.821999999999996
3760
+ - type: precision_at_1
3761
+ value: 33.955
3762
+ - type: precision_at_10
3763
+ value: 7.649
3764
+ - type: precision_at_100
3765
+ value: 1.173
3766
+ - type: precision_at_1000
3767
+ value: 0.14200000000000002
3768
+ - type: precision_at_3
3769
+ value: 17.817
3770
+ - type: precision_at_5
3771
+ value: 12.537
3772
+ - type: recall_at_1
3773
+ value: 28.923
3774
+ - type: recall_at_10
3775
+ value: 58.934
3776
+ - type: recall_at_100
3777
+ value: 81.809
3778
+ - type: recall_at_1000
3779
+ value: 94.71300000000001
3780
+ - type: recall_at_3
3781
+ value: 42.975
3782
+ - type: recall_at_5
3783
+ value: 49.501
3784
+ - task:
3785
+ type: Retrieval
3786
+ dataset:
3787
+ type: mteb/cqadupstack-webmasters
3788
+ name: MTEB CQADupstackWebmastersRetrieval
3789
+ config: default
3790
+ split: test
3791
+ revision: 160c094312a0e1facb97e55eeddb698c0abe3571
3792
+ metrics:
3793
+ - type: map_at_1
3794
+ value: 28.596
3795
+ - type: map_at_10
3796
+ value: 38.735
3797
+ - type: map_at_100
3798
+ value: 40.264
3799
+ - type: map_at_1000
3800
+ value: 40.48
3801
+ - type: map_at_3
3802
+ value: 35.394999999999996
3803
+ - type: map_at_5
3804
+ value: 37.099
3805
+ - type: mrr_at_1
3806
+ value: 33.992
3807
+ - type: mrr_at_10
3808
+ value: 43.076
3809
+ - type: mrr_at_100
3810
+ value: 44.005
3811
+ - type: mrr_at_1000
3812
+ value: 44.043
3813
+ - type: mrr_at_3
3814
+ value: 40.415
3815
+ - type: mrr_at_5
3816
+ value: 41.957
3817
+ - type: ndcg_at_1
3818
+ value: 33.992
3819
+ - type: ndcg_at_10
3820
+ value: 44.896
3821
+ - type: ndcg_at_100
3822
+ value: 50.44499999999999
3823
+ - type: ndcg_at_1000
3824
+ value: 52.675000000000004
3825
+ - type: ndcg_at_3
3826
+ value: 39.783
3827
+ - type: ndcg_at_5
3828
+ value: 41.997
3829
+ - type: precision_at_1
3830
+ value: 33.992
3831
+ - type: precision_at_10
3832
+ value: 8.498
3833
+ - type: precision_at_100
3834
+ value: 1.585
3835
+ - type: precision_at_1000
3836
+ value: 0.248
3837
+ - type: precision_at_3
3838
+ value: 18.511
3839
+ - type: precision_at_5
3840
+ value: 13.241
3841
+ - type: recall_at_1
3842
+ value: 28.596
3843
+ - type: recall_at_10
3844
+ value: 56.885
3845
+ - type: recall_at_100
3846
+ value: 82.306
3847
+ - type: recall_at_1000
3848
+ value: 95.813
3849
+ - type: recall_at_3
3850
+ value: 42.168
3851
+ - type: recall_at_5
3852
+ value: 48.32
3853
+ - task:
3854
+ type: Retrieval
3855
+ dataset:
3856
+ type: mteb/cqadupstack-wordpress
3857
+ name: MTEB CQADupstackWordpressRetrieval
3858
+ config: default
3859
+ split: test
3860
+ revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
3861
+ metrics:
3862
+ - type: map_at_1
3863
+ value: 25.576
3864
+ - type: map_at_10
3865
+ value: 33.034
3866
+ - type: map_at_100
3867
+ value: 34.117999999999995
3868
+ - type: map_at_1000
3869
+ value: 34.222
3870
+ - type: map_at_3
3871
+ value: 30.183
3872
+ - type: map_at_5
3873
+ value: 31.974000000000004
3874
+ - type: mrr_at_1
3875
+ value: 27.542
3876
+ - type: mrr_at_10
3877
+ value: 34.838
3878
+ - type: mrr_at_100
3879
+ value: 35.824
3880
+ - type: mrr_at_1000
3881
+ value: 35.899
3882
+ - type: mrr_at_3
3883
+ value: 32.348
3884
+ - type: mrr_at_5
3885
+ value: 34.039
3886
+ - type: ndcg_at_1
3887
+ value: 27.542
3888
+ - type: ndcg_at_10
3889
+ value: 37.663000000000004
3890
+ - type: ndcg_at_100
3891
+ value: 42.762
3892
+ - type: ndcg_at_1000
3893
+ value: 45.235
3894
+ - type: ndcg_at_3
3895
+ value: 32.227
3896
+ - type: ndcg_at_5
3897
+ value: 35.27
3898
+ - type: precision_at_1
3899
+ value: 27.542
3900
+ - type: precision_at_10
3901
+ value: 5.840999999999999
3902
+ - type: precision_at_100
3903
+ value: 0.895
3904
+ - type: precision_at_1000
3905
+ value: 0.123
3906
+ - type: precision_at_3
3907
+ value: 13.370000000000001
3908
+ - type: precision_at_5
3909
+ value: 9.797
3910
+ - type: recall_at_1
3911
+ value: 25.576
3912
+ - type: recall_at_10
3913
+ value: 50.285000000000004
3914
+ - type: recall_at_100
3915
+ value: 73.06
3916
+ - type: recall_at_1000
3917
+ value: 91.15299999999999
3918
+ - type: recall_at_3
3919
+ value: 35.781
3920
+ - type: recall_at_5
3921
+ value: 43.058
3922
+ - task:
3923
+ type: Retrieval
3924
+ dataset:
3925
+ type: mteb/climate-fever
3926
+ name: MTEB ClimateFEVER
3927
+ config: default
3928
+ split: test
3929
+ revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
3930
+ metrics:
3931
+ - type: map_at_1
3932
+ value: 17.061
3933
+ - type: map_at_10
3934
+ value: 29.464000000000002
3935
+ - type: map_at_100
3936
+ value: 31.552999999999997
3937
+ - type: map_at_1000
3938
+ value: 31.707
3939
+ - type: map_at_3
3940
+ value: 24.834999999999997
3941
+ - type: map_at_5
3942
+ value: 27.355
3943
+ - type: mrr_at_1
3944
+ value: 38.958
3945
+ - type: mrr_at_10
3946
+ value: 51.578
3947
+ - type: mrr_at_100
3948
+ value: 52.262
3949
+ - type: mrr_at_1000
3950
+ value: 52.283
3951
+ - type: mrr_at_3
3952
+ value: 48.599
3953
+ - type: mrr_at_5
3954
+ value: 50.404
3955
+ - type: ndcg_at_1
3956
+ value: 38.958
3957
+ - type: ndcg_at_10
3958
+ value: 39.367999999999995
3959
+ - type: ndcg_at_100
3960
+ value: 46.521
3961
+ - type: ndcg_at_1000
3962
+ value: 49.086999999999996
3963
+ - type: ndcg_at_3
3964
+ value: 33.442
3965
+ - type: ndcg_at_5
3966
+ value: 35.515
3967
+ - type: precision_at_1
3968
+ value: 38.958
3969
+ - type: precision_at_10
3970
+ value: 12.110999999999999
3971
+ - type: precision_at_100
3972
+ value: 1.982
3973
+ - type: precision_at_1000
3974
+ value: 0.247
3975
+ - type: precision_at_3
3976
+ value: 25.102999999999998
3977
+ - type: precision_at_5
3978
+ value: 18.971
3979
+ - type: recall_at_1
3980
+ value: 17.061
3981
+ - type: recall_at_10
3982
+ value: 45.198
3983
+ - type: recall_at_100
3984
+ value: 69.18900000000001
3985
+ - type: recall_at_1000
3986
+ value: 83.38499999999999
3987
+ - type: recall_at_3
3988
+ value: 30.241
3989
+ - type: recall_at_5
3990
+ value: 36.851
3991
+ - task:
3992
+ type: Retrieval
3993
+ dataset:
3994
+ type: mteb/dbpedia
3995
+ name: MTEB DBPedia
3996
+ config: default
3997
+ split: test
3998
+ revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
3999
+ metrics:
4000
+ - type: map_at_1
4001
+ value: 9.398
4002
+ - type: map_at_10
4003
+ value: 21.421
4004
+ - type: map_at_100
4005
+ value: 31.649
4006
+ - type: map_at_1000
4007
+ value: 33.469
4008
+ - type: map_at_3
4009
+ value: 15.310000000000002
4010
+ - type: map_at_5
4011
+ value: 17.946
4012
+ - type: mrr_at_1
4013
+ value: 71
4014
+ - type: mrr_at_10
4015
+ value: 78.92099999999999
4016
+ - type: mrr_at_100
4017
+ value: 79.225
4018
+ - type: mrr_at_1000
4019
+ value: 79.23
4020
+ - type: mrr_at_3
4021
+ value: 77.792
4022
+ - type: mrr_at_5
4023
+ value: 78.467
4024
+ - type: ndcg_at_1
4025
+ value: 57.99999999999999
4026
+ - type: ndcg_at_10
4027
+ value: 44.733000000000004
4028
+ - type: ndcg_at_100
4029
+ value: 50.646
4030
+ - type: ndcg_at_1000
4031
+ value: 57.903999999999996
4032
+ - type: ndcg_at_3
4033
+ value: 49.175999999999995
4034
+ - type: ndcg_at_5
4035
+ value: 46.800999999999995
4036
+ - type: precision_at_1
4037
+ value: 71
4038
+ - type: precision_at_10
4039
+ value: 36.25
4040
+ - type: precision_at_100
4041
+ value: 12.135
4042
+ - type: precision_at_1000
4043
+ value: 2.26
4044
+ - type: precision_at_3
4045
+ value: 52.75
4046
+ - type: precision_at_5
4047
+ value: 45.65
4048
+ - type: recall_at_1
4049
+ value: 9.398
4050
+ - type: recall_at_10
4051
+ value: 26.596999999999998
4052
+ - type: recall_at_100
4053
+ value: 57.943
4054
+ - type: recall_at_1000
4055
+ value: 81.147
4056
+ - type: recall_at_3
4057
+ value: 16.634
4058
+ - type: recall_at_5
4059
+ value: 20.7
4060
+ - task:
4061
+ type: Classification
4062
+ dataset:
4063
+ type: mteb/emotion
4064
+ name: MTEB EmotionClassification
4065
+ config: default
4066
+ split: test
4067
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
4068
+ metrics:
4069
+ - type: accuracy
4070
+ value: 46.535000000000004
4071
+ - type: f1
4072
+ value: 42.53702746452163
4073
+ - task:
4074
+ type: Retrieval
4075
+ dataset:
4076
+ type: mteb/fever
4077
+ name: MTEB FEVER
4078
+ config: default
4079
+ split: test
4080
+ revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
4081
+ metrics:
4082
+ - type: map_at_1
4083
+ value: 77.235
4084
+ - type: map_at_10
4085
+ value: 85.504
4086
+ - type: map_at_100
4087
+ value: 85.707
4088
+ - type: map_at_1000
4089
+ value: 85.718
4090
+ - type: map_at_3
4091
+ value: 84.425
4092
+ - type: map_at_5
4093
+ value: 85.13
4094
+ - type: mrr_at_1
4095
+ value: 83.363
4096
+ - type: mrr_at_10
4097
+ value: 89.916
4098
+ - type: mrr_at_100
4099
+ value: 89.955
4100
+ - type: mrr_at_1000
4101
+ value: 89.956
4102
+ - type: mrr_at_3
4103
+ value: 89.32600000000001
4104
+ - type: mrr_at_5
4105
+ value: 89.79
4106
+ - type: ndcg_at_1
4107
+ value: 83.363
4108
+ - type: ndcg_at_10
4109
+ value: 89.015
4110
+ - type: ndcg_at_100
4111
+ value: 89.649
4112
+ - type: ndcg_at_1000
4113
+ value: 89.825
4114
+ - type: ndcg_at_3
4115
+ value: 87.45100000000001
4116
+ - type: ndcg_at_5
4117
+ value: 88.39399999999999
4118
+ - type: precision_at_1
4119
+ value: 83.363
4120
+ - type: precision_at_10
4121
+ value: 10.659
4122
+ - type: precision_at_100
4123
+ value: 1.122
4124
+ - type: precision_at_1000
4125
+ value: 0.11499999999999999
4126
+ - type: precision_at_3
4127
+ value: 33.338
4128
+ - type: precision_at_5
4129
+ value: 20.671999999999997
4130
+ - type: recall_at_1
4131
+ value: 77.235
4132
+ - type: recall_at_10
4133
+ value: 95.389
4134
+ - type: recall_at_100
4135
+ value: 97.722
4136
+ - type: recall_at_1000
4137
+ value: 98.744
4138
+ - type: recall_at_3
4139
+ value: 91.19800000000001
4140
+ - type: recall_at_5
4141
+ value: 93.635
4142
+ - task:
4143
+ type: Retrieval
4144
+ dataset:
4145
+ type: mteb/fiqa
4146
+ name: MTEB FiQA2018
4147
+ config: default
4148
+ split: test
4149
+ revision: 27a168819829fe9bcd655c2df245fb19452e8e06
4150
+ metrics:
4151
+ - type: map_at_1
4152
+ value: 20.835
4153
+ - type: map_at_10
4154
+ value: 34.459
4155
+ - type: map_at_100
4156
+ value: 36.335
4157
+ - type: map_at_1000
4158
+ value: 36.518
4159
+ - type: map_at_3
4160
+ value: 30.581000000000003
4161
+ - type: map_at_5
4162
+ value: 32.859
4163
+ - type: mrr_at_1
4164
+ value: 40.894999999999996
4165
+ - type: mrr_at_10
4166
+ value: 50.491
4167
+ - type: mrr_at_100
4168
+ value: 51.243
4169
+ - type: mrr_at_1000
4170
+ value: 51.286
4171
+ - type: mrr_at_3
4172
+ value: 47.994
4173
+ - type: mrr_at_5
4174
+ value: 49.429
4175
+ - type: ndcg_at_1
4176
+ value: 40.894999999999996
4177
+ - type: ndcg_at_10
4178
+ value: 42.403
4179
+ - type: ndcg_at_100
4180
+ value: 48.954
4181
+ - type: ndcg_at_1000
4182
+ value: 51.961
4183
+ - type: ndcg_at_3
4184
+ value: 39.11
4185
+ - type: ndcg_at_5
4186
+ value: 40.152
4187
+ - type: precision_at_1
4188
+ value: 40.894999999999996
4189
+ - type: precision_at_10
4190
+ value: 11.466
4191
+ - type: precision_at_100
4192
+ value: 1.833
4193
+ - type: precision_at_1000
4194
+ value: 0.23700000000000002
4195
+ - type: precision_at_3
4196
+ value: 25.874000000000002
4197
+ - type: precision_at_5
4198
+ value: 19.012
4199
+ - type: recall_at_1
4200
+ value: 20.835
4201
+ - type: recall_at_10
4202
+ value: 49.535000000000004
4203
+ - type: recall_at_100
4204
+ value: 73.39099999999999
4205
+ - type: recall_at_1000
4206
+ value: 91.01599999999999
4207
+ - type: recall_at_3
4208
+ value: 36.379
4209
+ - type: recall_at_5
4210
+ value: 42.059999999999995
4211
+ - task:
4212
+ type: Retrieval
4213
+ dataset:
4214
+ type: mteb/hotpotqa
4215
+ name: MTEB HotpotQA
4216
+ config: default
4217
+ split: test
4218
+ revision: ab518f4d6fcca38d87c25209f94beba119d02014
4219
+ metrics:
4220
+ - type: map_at_1
4221
+ value: 40.945
4222
+ - type: map_at_10
4223
+ value: 65.376
4224
+ - type: map_at_100
4225
+ value: 66.278
4226
+ - type: map_at_1000
4227
+ value: 66.33
4228
+ - type: map_at_3
4229
+ value: 61.753
4230
+ - type: map_at_5
4231
+ value: 64.077
4232
+ - type: mrr_at_1
4233
+ value: 81.891
4234
+ - type: mrr_at_10
4235
+ value: 87.256
4236
+ - type: mrr_at_100
4237
+ value: 87.392
4238
+ - type: mrr_at_1000
4239
+ value: 87.395
4240
+ - type: mrr_at_3
4241
+ value: 86.442
4242
+ - type: mrr_at_5
4243
+ value: 86.991
4244
+ - type: ndcg_at_1
4245
+ value: 81.891
4246
+ - type: ndcg_at_10
4247
+ value: 73.654
4248
+ - type: ndcg_at_100
4249
+ value: 76.62299999999999
4250
+ - type: ndcg_at_1000
4251
+ value: 77.60000000000001
4252
+ - type: ndcg_at_3
4253
+ value: 68.71199999999999
4254
+ - type: ndcg_at_5
4255
+ value: 71.563
4256
+ - type: precision_at_1
4257
+ value: 81.891
4258
+ - type: precision_at_10
4259
+ value: 15.409
4260
+ - type: precision_at_100
4261
+ value: 1.77
4262
+ - type: precision_at_1000
4263
+ value: 0.19
4264
+ - type: precision_at_3
4265
+ value: 44.15
4266
+ - type: precision_at_5
4267
+ value: 28.732000000000003
4268
+ - type: recall_at_1
4269
+ value: 40.945
4270
+ - type: recall_at_10
4271
+ value: 77.04299999999999
4272
+ - type: recall_at_100
4273
+ value: 88.508
4274
+ - type: recall_at_1000
4275
+ value: 94.943
4276
+ - type: recall_at_3
4277
+ value: 66.226
4278
+ - type: recall_at_5
4279
+ value: 71.83
4280
+ - task:
4281
+ type: Classification
4282
+ dataset:
4283
+ type: mteb/imdb
4284
+ name: MTEB ImdbClassification
4285
+ config: default
4286
+ split: test
4287
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
4288
+ metrics:
4289
+ - type: accuracy
4290
+ value: 74.08200000000001
4291
+ - type: ap
4292
+ value: 68.10929101713998
4293
+ - type: f1
4294
+ value: 73.98447117652009
4295
+ - task:
4296
+ type: Retrieval
4297
+ dataset:
4298
+ type: mteb/msmarco
4299
+ name: MTEB MSMARCO
4300
+ config: default
4301
+ split: dev
4302
+ revision: c5a29a104738b98a9e76336939199e264163d4a0
4303
+ metrics:
4304
+ - type: map_at_1
4305
+ value: 21.729000000000003
4306
+ - type: map_at_10
4307
+ value: 34.602
4308
+ - type: map_at_100
4309
+ value: 35.756
4310
+ - type: map_at_1000
4311
+ value: 35.803000000000004
4312
+ - type: map_at_3
4313
+ value: 30.619000000000003
4314
+ - type: map_at_5
4315
+ value: 32.914
4316
+ - type: mrr_at_1
4317
+ value: 22.364
4318
+ - type: mrr_at_10
4319
+ value: 35.183
4320
+ - type: mrr_at_100
4321
+ value: 36.287000000000006
4322
+ - type: mrr_at_1000
4323
+ value: 36.327999999999996
4324
+ - type: mrr_at_3
4325
+ value: 31.258000000000003
4326
+ - type: mrr_at_5
4327
+ value: 33.542
4328
+ - type: ndcg_at_1
4329
+ value: 22.364
4330
+ - type: ndcg_at_10
4331
+ value: 41.765
4332
+ - type: ndcg_at_100
4333
+ value: 47.293
4334
+ - type: ndcg_at_1000
4335
+ value: 48.457
4336
+ - type: ndcg_at_3
4337
+ value: 33.676
4338
+ - type: ndcg_at_5
4339
+ value: 37.783
4340
+ - type: precision_at_1
4341
+ value: 22.364
4342
+ - type: precision_at_10
4343
+ value: 6.662
4344
+ - type: precision_at_100
4345
+ value: 0.943
4346
+ - type: precision_at_1000
4347
+ value: 0.104
4348
+ - type: precision_at_3
4349
+ value: 14.435999999999998
4350
+ - type: precision_at_5
4351
+ value: 10.764999999999999
4352
+ - type: recall_at_1
4353
+ value: 21.729000000000003
4354
+ - type: recall_at_10
4355
+ value: 63.815999999999995
4356
+ - type: recall_at_100
4357
+ value: 89.265
4358
+ - type: recall_at_1000
4359
+ value: 98.149
4360
+ - type: recall_at_3
4361
+ value: 41.898
4362
+ - type: recall_at_5
4363
+ value: 51.76500000000001
4364
+ - task:
4365
+ type: Classification
4366
+ dataset:
4367
+ type: mteb/mtop_domain
4368
+ name: MTEB MTOPDomainClassification (en)
4369
+ config: en
4370
+ split: test
4371
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
4372
+ metrics:
4373
+ - type: accuracy
4374
+ value: 92.73141814865483
4375
+ - type: f1
4376
+ value: 92.17518476408004
4377
+ - task:
4378
+ type: Classification
4379
+ dataset:
4380
+ type: mteb/mtop_intent
4381
+ name: MTEB MTOPIntentClassification (en)
4382
+ config: en
4383
+ split: test
4384
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
4385
+ metrics:
4386
+ - type: accuracy
4387
+ value: 65.18011855905152
4388
+ - type: f1
4389
+ value: 46.70999638311856
4390
+ - task:
4391
+ type: Classification
4392
+ dataset:
4393
+ type: masakhane/masakhanews
4394
+ name: MTEB MasakhaNEWSClassification (eng)
4395
+ config: eng
4396
+ split: test
4397
+ revision: 8ccc72e69e65f40c70e117d8b3c08306bb788b60
4398
+ metrics:
4399
+ - type: accuracy
4400
+ value: 75.24261603375525
4401
+ - type: f1
4402
+ value: 74.07895183913367
4403
+ - task:
4404
+ type: Clustering
4405
+ dataset:
4406
+ type: masakhane/masakhanews
4407
+ name: MTEB MasakhaNEWSClusteringP2P (eng)
4408
+ config: eng
4409
+ split: test
4410
+ revision: 8ccc72e69e65f40c70e117d8b3c08306bb788b60
4411
+ metrics:
4412
+ - type: v_measure
4413
+ value: 28.43855875387446
4414
+ - task:
4415
+ type: Clustering
4416
+ dataset:
4417
+ type: masakhane/masakhanews
4418
+ name: MTEB MasakhaNEWSClusteringS2S (eng)
4419
+ config: eng
4420
+ split: test
4421
+ revision: 8ccc72e69e65f40c70e117d8b3c08306bb788b60
4422
+ metrics:
4423
+ - type: v_measure
4424
+ value: 29.05331990256969
4425
+ - task:
4426
+ type: Classification
4427
+ dataset:
4428
+ type: mteb/amazon_massive_intent
4429
+ name: MTEB MassiveIntentClassification (en)
4430
+ config: en
4431
+ split: test
4432
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
4433
+ metrics:
4434
+ - type: accuracy
4435
+ value: 66.92333557498318
4436
+ - type: f1
4437
+ value: 64.29789389602692
4438
+ - task:
4439
+ type: Classification
4440
+ dataset:
4441
+ type: mteb/amazon_massive_scenario
4442
+ name: MTEB MassiveScenarioClassification (en)
4443
+ config: en
4444
+ split: test
4445
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
4446
+ metrics:
4447
+ - type: accuracy
4448
+ value: 72.74714189643578
4449
+ - type: f1
4450
+ value: 71.672585608315
4451
+ - task:
4452
+ type: Clustering
4453
+ dataset:
4454
+ type: mteb/medrxiv-clustering-p2p
4455
+ name: MTEB MedrxivClusteringP2P
4456
+ config: default
4457
+ split: test
4458
+ revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
4459
+ metrics:
4460
+ - type: v_measure
4461
+ value: 31.503564225501613
4462
+ - task:
4463
+ type: Clustering
4464
+ dataset:
4465
+ type: mteb/medrxiv-clustering-s2s
4466
+ name: MTEB MedrxivClusteringS2S
4467
+ config: default
4468
+ split: test
4469
+ revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
4470
+ metrics:
4471
+ - type: v_measure
4472
+ value: 28.410225127136457
4473
+ - task:
4474
+ type: Reranking
4475
+ dataset:
4476
+ type: mteb/mind_small
4477
+ name: MTEB MindSmallReranking
4478
+ config: default
4479
+ split: test
4480
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
4481
+ metrics:
4482
+ - type: map
4483
+ value: 29.170019896091908
4484
+ - type: mrr
4485
+ value: 29.881276831500976
4486
+ - task:
4487
+ type: Retrieval
4488
+ dataset:
4489
+ type: mteb/nfcorpus
4490
+ name: MTEB NFCorpus
4491
+ config: default
4492
+ split: test
4493
+ revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
4494
+ metrics:
4495
+ - type: map_at_1
4496
+ value: 6.544
4497
+ - type: map_at_10
4498
+ value: 14.116999999999999
4499
+ - type: map_at_100
4500
+ value: 17.522
4501
+ - type: map_at_1000
4502
+ value: 19
4503
+ - type: map_at_3
4504
+ value: 10.369
4505
+ - type: map_at_5
4506
+ value: 12.189
4507
+ - type: mrr_at_1
4508
+ value: 47.988
4509
+ - type: mrr_at_10
4510
+ value: 56.84
4511
+ - type: mrr_at_100
4512
+ value: 57.367000000000004
4513
+ - type: mrr_at_1000
4514
+ value: 57.403000000000006
4515
+ - type: mrr_at_3
4516
+ value: 54.592
4517
+ - type: mrr_at_5
4518
+ value: 56.233
4519
+ - type: ndcg_at_1
4520
+ value: 45.82
4521
+ - type: ndcg_at_10
4522
+ value: 36.767
4523
+ - type: ndcg_at_100
4524
+ value: 33.356
4525
+ - type: ndcg_at_1000
4526
+ value: 42.062
4527
+ - type: ndcg_at_3
4528
+ value: 42.15
4529
+ - type: ndcg_at_5
4530
+ value: 40.355000000000004
4531
+ - type: precision_at_1
4532
+ value: 47.988
4533
+ - type: precision_at_10
4534
+ value: 27.121000000000002
4535
+ - type: precision_at_100
4536
+ value: 8.455
4537
+ - type: precision_at_1000
4538
+ value: 2.103
4539
+ - type: precision_at_3
4540
+ value: 39.628
4541
+ - type: precision_at_5
4542
+ value: 35.356
4543
+ - type: recall_at_1
4544
+ value: 6.544
4545
+ - type: recall_at_10
4546
+ value: 17.928
4547
+ - type: recall_at_100
4548
+ value: 32.843
4549
+ - type: recall_at_1000
4550
+ value: 65.752
4551
+ - type: recall_at_3
4552
+ value: 11.297
4553
+ - type: recall_at_5
4554
+ value: 14.357000000000001
4555
+ - task:
4556
+ type: Retrieval
4557
+ dataset:
4558
+ type: mteb/nq
4559
+ name: MTEB NQ
4560
+ config: default
4561
+ split: test
4562
+ revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
4563
+ metrics:
4564
+ - type: map_at_1
4565
+ value: 39.262
4566
+ - type: map_at_10
4567
+ value: 55.095000000000006
4568
+ - type: map_at_100
4569
+ value: 55.93900000000001
4570
+ - type: map_at_1000
4571
+ value: 55.955999999999996
4572
+ - type: map_at_3
4573
+ value: 50.93
4574
+ - type: map_at_5
4575
+ value: 53.491
4576
+ - type: mrr_at_1
4577
+ value: 43.598
4578
+ - type: mrr_at_10
4579
+ value: 57.379999999999995
4580
+ - type: mrr_at_100
4581
+ value: 57.940999999999995
4582
+ - type: mrr_at_1000
4583
+ value: 57.952000000000005
4584
+ - type: mrr_at_3
4585
+ value: 53.998000000000005
4586
+ - type: mrr_at_5
4587
+ value: 56.128
4588
+ - type: ndcg_at_1
4589
+ value: 43.598
4590
+ - type: ndcg_at_10
4591
+ value: 62.427
4592
+ - type: ndcg_at_100
4593
+ value: 65.759
4594
+ - type: ndcg_at_1000
4595
+ value: 66.133
4596
+ - type: ndcg_at_3
4597
+ value: 54.745999999999995
4598
+ - type: ndcg_at_5
4599
+ value: 58.975
4600
+ - type: precision_at_1
4601
+ value: 43.598
4602
+ - type: precision_at_10
4603
+ value: 9.789
4604
+ - type: precision_at_100
4605
+ value: 1.171
4606
+ - type: precision_at_1000
4607
+ value: 0.121
4608
+ - type: precision_at_3
4609
+ value: 24.295
4610
+ - type: precision_at_5
4611
+ value: 17.028
4612
+ - type: recall_at_1
4613
+ value: 39.262
4614
+ - type: recall_at_10
4615
+ value: 82.317
4616
+ - type: recall_at_100
4617
+ value: 96.391
4618
+ - type: recall_at_1000
4619
+ value: 99.116
4620
+ - type: recall_at_3
4621
+ value: 62.621
4622
+ - type: recall_at_5
4623
+ value: 72.357
4624
+ - task:
4625
+ type: Classification
4626
+ dataset:
4627
+ type: ag_news
4628
+ name: MTEB NewsClassification
4629
+ config: default
4630
+ split: test
4631
+ revision: eb185aade064a813bc0b7f42de02595523103ca4
4632
+ metrics:
4633
+ - type: accuracy
4634
+ value: 78.17500000000001
4635
+ - type: f1
4636
+ value: 78.01940892857273
4637
+ - task:
4638
+ type: PairClassification
4639
+ dataset:
4640
+ type: GEM/opusparcus
4641
+ name: MTEB OpusparcusPC (en)
4642
+ config: en
4643
+ split: test
4644
+ revision: 9e9b1f8ef51616073f47f306f7f47dd91663f86a
4645
+ metrics:
4646
+ - type: cos_sim_accuracy
4647
+ value: 99.89816700610999
4648
+ - type: cos_sim_ap
4649
+ value: 100
4650
+ - type: cos_sim_f1
4651
+ value: 99.9490575649516
4652
+ - type: cos_sim_precision
4653
+ value: 100
4654
+ - type: cos_sim_recall
4655
+ value: 99.89816700610999
4656
+ - type: dot_accuracy
4657
+ value: 99.89816700610999
4658
+ - type: dot_ap
4659
+ value: 100
4660
+ - type: dot_f1
4661
+ value: 99.9490575649516
4662
+ - type: dot_precision
4663
+ value: 100
4664
+ - type: dot_recall
4665
+ value: 99.89816700610999
4666
+ - type: euclidean_accuracy
4667
+ value: 99.89816700610999
4668
+ - type: euclidean_ap
4669
+ value: 100
4670
+ - type: euclidean_f1
4671
+ value: 99.9490575649516
4672
+ - type: euclidean_precision
4673
+ value: 100
4674
+ - type: euclidean_recall
4675
+ value: 99.89816700610999
4676
+ - type: manhattan_accuracy
4677
+ value: 99.89816700610999
4678
+ - type: manhattan_ap
4679
+ value: 100
4680
+ - type: manhattan_f1
4681
+ value: 99.9490575649516
4682
+ - type: manhattan_precision
4683
+ value: 100
4684
+ - type: manhattan_recall
4685
+ value: 99.89816700610999
4686
+ - type: max_accuracy
4687
+ value: 99.89816700610999
4688
+ - type: max_ap
4689
+ value: 100
4690
+ - type: max_f1
4691
+ value: 99.9490575649516
4692
+ - task:
4693
+ type: PairClassification
4694
+ dataset:
4695
+ type: paws-x
4696
+ name: MTEB PawsX (en)
4697
+ config: en
4698
+ split: test
4699
+ revision: 8a04d940a42cd40658986fdd8e3da561533a3646
4700
+ metrics:
4701
+ - type: cos_sim_accuracy
4702
+ value: 61
4703
+ - type: cos_sim_ap
4704
+ value: 59.630757252602464
4705
+ - type: cos_sim_f1
4706
+ value: 62.37521514629949
4707
+ - type: cos_sim_precision
4708
+ value: 45.34534534534534
4709
+ - type: cos_sim_recall
4710
+ value: 99.88974641675854
4711
+ - type: dot_accuracy
4712
+ value: 61
4713
+ - type: dot_ap
4714
+ value: 59.631527308059006
4715
+ - type: dot_f1
4716
+ value: 62.37521514629949
4717
+ - type: dot_precision
4718
+ value: 45.34534534534534
4719
+ - type: dot_recall
4720
+ value: 99.88974641675854
4721
+ - type: euclidean_accuracy
4722
+ value: 61
4723
+ - type: euclidean_ap
4724
+ value: 59.630757252602464
4725
+ - type: euclidean_f1
4726
+ value: 62.37521514629949
4727
+ - type: euclidean_precision
4728
+ value: 45.34534534534534
4729
+ - type: euclidean_recall
4730
+ value: 99.88974641675854
4731
+ - type: manhattan_accuracy
4732
+ value: 60.9
4733
+ - type: manhattan_ap
4734
+ value: 59.613947780462254
4735
+ - type: manhattan_f1
4736
+ value: 62.37521514629949
4737
+ - type: manhattan_precision
4738
+ value: 45.34534534534534
4739
+ - type: manhattan_recall
4740
+ value: 99.88974641675854
4741
+ - type: max_accuracy
4742
+ value: 61
4743
+ - type: max_ap
4744
+ value: 59.631527308059006
4745
+ - type: max_f1
4746
+ value: 62.37521514629949
4747
+ - task:
4748
+ type: Retrieval
4749
+ dataset:
4750
+ type: mteb/quora
4751
+ name: MTEB QuoraRetrieval
4752
+ config: default
4753
+ split: test
4754
+ revision: e4e08e0b7dbe3c8700f0daef558ff32256715259
4755
+ metrics:
4756
+ - type: map_at_1
4757
+ value: 69.963
4758
+ - type: map_at_10
4759
+ value: 83.59400000000001
4760
+ - type: map_at_100
4761
+ value: 84.236
4762
+ - type: map_at_1000
4763
+ value: 84.255
4764
+ - type: map_at_3
4765
+ value: 80.69800000000001
4766
+ - type: map_at_5
4767
+ value: 82.568
4768
+ - type: mrr_at_1
4769
+ value: 80.58999999999999
4770
+ - type: mrr_at_10
4771
+ value: 86.78200000000001
4772
+ - type: mrr_at_100
4773
+ value: 86.89099999999999
4774
+ - type: mrr_at_1000
4775
+ value: 86.893
4776
+ - type: mrr_at_3
4777
+ value: 85.757
4778
+ - type: mrr_at_5
4779
+ value: 86.507
4780
+ - type: ndcg_at_1
4781
+ value: 80.60000000000001
4782
+ - type: ndcg_at_10
4783
+ value: 87.41799999999999
4784
+ - type: ndcg_at_100
4785
+ value: 88.723
4786
+ - type: ndcg_at_1000
4787
+ value: 88.875
4788
+ - type: ndcg_at_3
4789
+ value: 84.565
4790
+ - type: ndcg_at_5
4791
+ value: 86.236
4792
+ - type: precision_at_1
4793
+ value: 80.60000000000001
4794
+ - type: precision_at_10
4795
+ value: 13.239
4796
+ - type: precision_at_100
4797
+ value: 1.5150000000000001
4798
+ - type: precision_at_1000
4799
+ value: 0.156
4800
+ - type: precision_at_3
4801
+ value: 36.947
4802
+ - type: precision_at_5
4803
+ value: 24.354
4804
+ - type: recall_at_1
4805
+ value: 69.963
4806
+ - type: recall_at_10
4807
+ value: 94.553
4808
+ - type: recall_at_100
4809
+ value: 99.104
4810
+ - type: recall_at_1000
4811
+ value: 99.872
4812
+ - type: recall_at_3
4813
+ value: 86.317
4814
+ - type: recall_at_5
4815
+ value: 91.023
4816
+ - task:
4817
+ type: Clustering
4818
+ dataset:
4819
+ type: mteb/reddit-clustering
4820
+ name: MTEB RedditClustering
4821
+ config: default
4822
+ split: test
4823
+ revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
4824
+ metrics:
4825
+ - type: v_measure
4826
+ value: 47.52890410998761
4827
+ - task:
4828
+ type: Clustering
4829
+ dataset:
4830
+ type: mteb/reddit-clustering-p2p
4831
+ name: MTEB RedditClusteringP2P
4832
+ config: default
4833
+ split: test
4834
+ revision: 385e3cb46b4cfa89021f56c4380204149d0efe33
4835
+ metrics:
4836
+ - type: v_measure
4837
+ value: 62.760692287940486
4838
+ - task:
4839
+ type: Retrieval
4840
+ dataset:
4841
+ type: mteb/scidocs
4842
+ name: MTEB SCIDOCS
4843
+ config: default
4844
+ split: test
4845
+ revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88
4846
+ metrics:
4847
+ - type: map_at_1
4848
+ value: 5.093
4849
+ - type: map_at_10
4850
+ value: 12.695
4851
+ - type: map_at_100
4852
+ value: 14.824000000000002
4853
+ - type: map_at_1000
4854
+ value: 15.123000000000001
4855
+ - type: map_at_3
4856
+ value: 8.968
4857
+ - type: map_at_5
4858
+ value: 10.828
4859
+ - type: mrr_at_1
4860
+ value: 25.1
4861
+ - type: mrr_at_10
4862
+ value: 35.894999999999996
4863
+ - type: mrr_at_100
4864
+ value: 36.966
4865
+ - type: mrr_at_1000
4866
+ value: 37.019999999999996
4867
+ - type: mrr_at_3
4868
+ value: 32.467
4869
+ - type: mrr_at_5
4870
+ value: 34.416999999999994
4871
+ - type: ndcg_at_1
4872
+ value: 25.1
4873
+ - type: ndcg_at_10
4874
+ value: 21.096999999999998
4875
+ - type: ndcg_at_100
4876
+ value: 29.202
4877
+ - type: ndcg_at_1000
4878
+ value: 34.541
4879
+ - type: ndcg_at_3
4880
+ value: 19.875
4881
+ - type: ndcg_at_5
4882
+ value: 17.497
4883
+ - type: precision_at_1
4884
+ value: 25.1
4885
+ - type: precision_at_10
4886
+ value: 10.9
4887
+ - type: precision_at_100
4888
+ value: 2.255
4889
+ - type: precision_at_1000
4890
+ value: 0.35400000000000004
4891
+ - type: precision_at_3
4892
+ value: 18.367
4893
+ - type: precision_at_5
4894
+ value: 15.299999999999999
4895
+ - type: recall_at_1
4896
+ value: 5.093
4897
+ - type: recall_at_10
4898
+ value: 22.092
4899
+ - type: recall_at_100
4900
+ value: 45.778
4901
+ - type: recall_at_1000
4902
+ value: 71.985
4903
+ - type: recall_at_3
4904
+ value: 11.167
4905
+ - type: recall_at_5
4906
+ value: 15.501999999999999
4907
+ - task:
4908
+ type: STS
4909
+ dataset:
4910
+ type: mteb/sickr-sts
4911
+ name: MTEB SICK-R
4912
+ config: default
4913
+ split: test
4914
+ revision: 20a6d6f312dd54037fe07a32d58e5e168867909d
4915
+ metrics:
4916
+ - type: cos_sim_pearson
4917
+ value: 74.04386981759481
4918
+ - type: cos_sim_spearman
4919
+ value: 69.12484963763646
4920
+ - type: euclidean_pearson
4921
+ value: 71.49384353291062
4922
+ - type: euclidean_spearman
4923
+ value: 69.12484548317074
4924
+ - type: manhattan_pearson
4925
+ value: 71.49828173987272
4926
+ - type: manhattan_spearman
4927
+ value: 69.08350274367014
4928
+ - task:
4929
+ type: STS
4930
+ dataset:
4931
+ type: mteb/sts12-sts
4932
+ name: MTEB STS12
4933
+ config: default
4934
+ split: test
4935
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
4936
+ metrics:
4937
+ - type: cos_sim_pearson
4938
+ value: 66.95372527615659
4939
+ - type: cos_sim_spearman
4940
+ value: 66.96821894433991
4941
+ - type: euclidean_pearson
4942
+ value: 64.675348002074
4943
+ - type: euclidean_spearman
4944
+ value: 66.96821894433991
4945
+ - type: manhattan_pearson
4946
+ value: 64.5965887073831
4947
+ - type: manhattan_spearman
4948
+ value: 66.88569076794741
4949
+ - task:
4950
+ type: STS
4951
+ dataset:
4952
+ type: mteb/sts13-sts
4953
+ name: MTEB STS13
4954
+ config: default
4955
+ split: test
4956
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
4957
+ metrics:
4958
+ - type: cos_sim_pearson
4959
+ value: 77.34698437961983
4960
+ - type: cos_sim_spearman
4961
+ value: 79.1153001117325
4962
+ - type: euclidean_pearson
4963
+ value: 78.53562874696966
4964
+ - type: euclidean_spearman
4965
+ value: 79.11530018205724
4966
+ - type: manhattan_pearson
4967
+ value: 78.46484988944093
4968
+ - type: manhattan_spearman
4969
+ value: 79.01416027493104
4970
+ - task:
4971
+ type: STS
4972
+ dataset:
4973
+ type: mteb/sts14-sts
4974
+ name: MTEB STS14
4975
+ config: default
4976
+ split: test
4977
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
4978
+ metrics:
4979
+ - type: cos_sim_pearson
4980
+ value: 68.81220371935373
4981
+ - type: cos_sim_spearman
4982
+ value: 68.50538405089604
4983
+ - type: euclidean_pearson
4984
+ value: 68.69204272683749
4985
+ - type: euclidean_spearman
4986
+ value: 68.50534223912419
4987
+ - type: manhattan_pearson
4988
+ value: 68.67300120149523
4989
+ - type: manhattan_spearman
4990
+ value: 68.45404301623115
4991
+ - task:
4992
+ type: STS
4993
+ dataset:
4994
+ type: mteb/sts15-sts
4995
+ name: MTEB STS15
4996
+ config: default
4997
+ split: test
4998
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
4999
+ metrics:
5000
+ - type: cos_sim_pearson
5001
+ value: 78.2464678879813
5002
+ - type: cos_sim_spearman
5003
+ value: 79.92003940566667
5004
+ - type: euclidean_pearson
5005
+ value: 79.8080778793964
5006
+ - type: euclidean_spearman
5007
+ value: 79.92003940566667
5008
+ - type: manhattan_pearson
5009
+ value: 79.80153621444681
5010
+ - type: manhattan_spearman
5011
+ value: 79.91293261418134
5012
+ - task:
5013
+ type: STS
5014
+ dataset:
5015
+ type: mteb/sts16-sts
5016
+ name: MTEB STS16
5017
+ config: default
5018
+ split: test
5019
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
5020
+ metrics:
5021
+ - type: cos_sim_pearson
5022
+ value: 76.31179207708662
5023
+ - type: cos_sim_spearman
5024
+ value: 78.65597349856115
5025
+ - type: euclidean_pearson
5026
+ value: 78.76937027472678
5027
+ - type: euclidean_spearman
5028
+ value: 78.65597349856115
5029
+ - type: manhattan_pearson
5030
+ value: 78.77129513300605
5031
+ - type: manhattan_spearman
5032
+ value: 78.62640467680775
5033
+ - task:
5034
+ type: STS
5035
+ dataset:
5036
+ type: mteb/sts17-crosslingual-sts
5037
+ name: MTEB STS17 (en-en)
5038
+ config: en-en
5039
+ split: test
5040
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
5041
+ metrics:
5042
+ - type: cos_sim_pearson
5043
+ value: 79.43158429552561
5044
+ - type: cos_sim_spearman
5045
+ value: 81.46108646565362
5046
+ - type: euclidean_pearson
5047
+ value: 81.47071791452292
5048
+ - type: euclidean_spearman
5049
+ value: 81.46108646565362
5050
+ - type: manhattan_pearson
5051
+ value: 81.56920643846031
5052
+ - type: manhattan_spearman
5053
+ value: 81.42226241399516
5054
+ - task:
5055
+ type: STS
5056
+ dataset:
5057
+ type: mteb/sts22-crosslingual-sts
5058
+ name: MTEB STS22 (en)
5059
+ config: en
5060
+ split: test
5061
+ revision: eea2b4fe26a775864c896887d910b76a8098ad3f
5062
+ metrics:
5063
+ - type: cos_sim_pearson
5064
+ value: 66.89546474141514
5065
+ - type: cos_sim_spearman
5066
+ value: 65.8393752170531
5067
+ - type: euclidean_pearson
5068
+ value: 67.2580522762307
5069
+ - type: euclidean_spearman
5070
+ value: 65.8393752170531
5071
+ - type: manhattan_pearson
5072
+ value: 67.45157729300522
5073
+ - type: manhattan_spearman
5074
+ value: 66.19470854403802
5075
+ - task:
5076
+ type: STS
5077
+ dataset:
5078
+ type: mteb/stsbenchmark-sts
5079
+ name: MTEB STSBenchmark
5080
+ config: default
5081
+ split: test
5082
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
5083
+ metrics:
5084
+ - type: cos_sim_pearson
5085
+ value: 71.39566306334434
5086
+ - type: cos_sim_spearman
5087
+ value: 74.0981396086974
5088
+ - type: euclidean_pearson
5089
+ value: 73.7834496259745
5090
+ - type: euclidean_spearman
5091
+ value: 74.09803741302046
5092
+ - type: manhattan_pearson
5093
+ value: 73.79958138780945
5094
+ - type: manhattan_spearman
5095
+ value: 74.09894837555905
5096
+ - task:
5097
+ type: STS
5098
+ dataset:
5099
+ type: PhilipMay/stsb_multi_mt
5100
+ name: MTEB STSBenchmarkMultilingualSTS (en)
5101
+ config: en
5102
+ split: test
5103
+ revision: 93d57ef91790589e3ce9c365164337a8a78b7632
5104
+ metrics:
5105
+ - type: cos_sim_pearson
5106
+ value: 71.39566311006806
5107
+ - type: cos_sim_spearman
5108
+ value: 74.0981396086974
5109
+ - type: euclidean_pearson
5110
+ value: 73.78344970897099
5111
+ - type: euclidean_spearman
5112
+ value: 74.09803741302046
5113
+ - type: manhattan_pearson
5114
+ value: 73.79958147136705
5115
+ - type: manhattan_spearman
5116
+ value: 74.09894837555905
5117
+ - task:
5118
+ type: Reranking
5119
+ dataset:
5120
+ type: mteb/scidocs-reranking
5121
+ name: MTEB SciDocsRR
5122
+ config: default
5123
+ split: test
5124
+ revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
5125
+ metrics:
5126
+ - type: map
5127
+ value: 80.81059564334683
5128
+ - type: mrr
5129
+ value: 94.62696617108381
5130
+ - task:
5131
+ type: Retrieval
5132
+ dataset:
5133
+ type: mteb/scifact
5134
+ name: MTEB SciFact
5135
+ config: default
5136
+ split: test
5137
+ revision: 0228b52cf27578f30900b9e5271d331663a030d7
5138
+ metrics:
5139
+ - type: map_at_1
5140
+ value: 57.760999999999996
5141
+ - type: map_at_10
5142
+ value: 68.614
5143
+ - type: map_at_100
5144
+ value: 69.109
5145
+ - type: map_at_1000
5146
+ value: 69.134
5147
+ - type: map_at_3
5148
+ value: 65.735
5149
+ - type: map_at_5
5150
+ value: 67.42099999999999
5151
+ - type: mrr_at_1
5152
+ value: 60.667
5153
+ - type: mrr_at_10
5154
+ value: 69.94200000000001
5155
+ - type: mrr_at_100
5156
+ value: 70.254
5157
+ - type: mrr_at_1000
5158
+ value: 70.28
5159
+ - type: mrr_at_3
5160
+ value: 67.72200000000001
5161
+ - type: mrr_at_5
5162
+ value: 69.18900000000001
5163
+ - type: ndcg_at_1
5164
+ value: 60.667
5165
+ - type: ndcg_at_10
5166
+ value: 73.548
5167
+ - type: ndcg_at_100
5168
+ value: 75.381
5169
+ - type: ndcg_at_1000
5170
+ value: 75.991
5171
+ - type: ndcg_at_3
5172
+ value: 68.685
5173
+ - type: ndcg_at_5
5174
+ value: 71.26
5175
+ - type: precision_at_1
5176
+ value: 60.667
5177
+ - type: precision_at_10
5178
+ value: 9.833
5179
+ - type: precision_at_100
5180
+ value: 1.08
5181
+ - type: precision_at_1000
5182
+ value: 0.11299999999999999
5183
+ - type: precision_at_3
5184
+ value: 26.889000000000003
5185
+ - type: precision_at_5
5186
+ value: 17.8
5187
+ - type: recall_at_1
5188
+ value: 57.760999999999996
5189
+ - type: recall_at_10
5190
+ value: 87.13300000000001
5191
+ - type: recall_at_100
5192
+ value: 95
5193
+ - type: recall_at_1000
5194
+ value: 99.667
5195
+ - type: recall_at_3
5196
+ value: 74.211
5197
+ - type: recall_at_5
5198
+ value: 80.63900000000001
5199
+ - task:
5200
+ type: PairClassification
5201
+ dataset:
5202
+ type: mteb/sprintduplicatequestions-pairclassification
5203
+ name: MTEB SprintDuplicateQuestions
5204
+ config: default
5205
+ split: test
5206
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
5207
+ metrics:
5208
+ - type: cos_sim_accuracy
5209
+ value: 99.81881188118813
5210
+ - type: cos_sim_ap
5211
+ value: 95.21196473745837
5212
+ - type: cos_sim_f1
5213
+ value: 90.69767441860465
5214
+ - type: cos_sim_precision
5215
+ value: 91.71779141104295
5216
+ - type: cos_sim_recall
5217
+ value: 89.7
5218
+ - type: dot_accuracy
5219
+ value: 99.81881188118813
5220
+ - type: dot_ap
5221
+ value: 95.21196473745837
5222
+ - type: dot_f1
5223
+ value: 90.69767441860465
5224
+ - type: dot_precision
5225
+ value: 91.71779141104295
5226
+ - type: dot_recall
5227
+ value: 89.7
5228
+ - type: euclidean_accuracy
5229
+ value: 99.81881188118813
5230
+ - type: euclidean_ap
5231
+ value: 95.21196473745839
5232
+ - type: euclidean_f1
5233
+ value: 90.69767441860465
5234
+ - type: euclidean_precision
5235
+ value: 91.71779141104295
5236
+ - type: euclidean_recall
5237
+ value: 89.7
5238
+ - type: manhattan_accuracy
5239
+ value: 99.81287128712871
5240
+ - type: manhattan_ap
5241
+ value: 95.16667174835017
5242
+ - type: manhattan_f1
5243
+ value: 90.41095890410959
5244
+ - type: manhattan_precision
5245
+ value: 91.7610710607621
5246
+ - type: manhattan_recall
5247
+ value: 89.1
5248
+ - type: max_accuracy
5249
+ value: 99.81881188118813
5250
+ - type: max_ap
5251
+ value: 95.21196473745839
5252
+ - type: max_f1
5253
+ value: 90.69767441860465
5254
+ - task:
5255
+ type: Clustering
5256
+ dataset:
5257
+ type: mteb/stackexchange-clustering
5258
+ name: MTEB StackExchangeClustering
5259
+ config: default
5260
+ split: test
5261
+ revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
5262
+ metrics:
5263
+ - type: v_measure
5264
+ value: 59.54942204515638
5265
+ - task:
5266
+ type: Clustering
5267
+ dataset:
5268
+ type: mteb/stackexchange-clustering-p2p
5269
+ name: MTEB StackExchangeClusteringP2P
5270
+ config: default
5271
+ split: test
5272
+ revision: 815ca46b2622cec33ccafc3735d572c266efdb44
5273
+ metrics:
5274
+ - type: v_measure
5275
+ value: 39.42892282672948
5276
+ - task:
5277
+ type: Reranking
5278
+ dataset:
5279
+ type: mteb/stackoverflowdupquestions-reranking
5280
+ name: MTEB StackOverflowDupQuestions
5281
+ config: default
5282
+ split: test
5283
+ revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
5284
+ metrics:
5285
+ - type: map
5286
+ value: 51.189033075914324
5287
+ - type: mrr
5288
+ value: 51.97014790764791
5289
+ - task:
5290
+ type: Summarization
5291
+ dataset:
5292
+ type: mteb/summeval
5293
+ name: MTEB SummEval
5294
+ config: default
5295
+ split: test
5296
+ revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
5297
+ metrics:
5298
+ - type: cos_sim_pearson
5299
+ value: 30.09466569775977
5300
+ - type: cos_sim_spearman
5301
+ value: 30.31058660775912
5302
+ - type: dot_pearson
5303
+ value: 30.09466438861689
5304
+ - type: dot_spearman
5305
+ value: 30.31058660775912
5306
+ - task:
5307
+ type: Retrieval
5308
+ dataset:
5309
+ type: mteb/trec-covid
5310
+ name: MTEB TRECCOVID
5311
+ config: default
5312
+ split: test
5313
+ revision: bb9466bac8153a0349341eb1b22e06409e78ef4e
5314
+ metrics:
5315
+ - type: map_at_1
5316
+ value: 0.253
5317
+ - type: map_at_10
5318
+ value: 2.07
5319
+ - type: map_at_100
5320
+ value: 12.679000000000002
5321
+ - type: map_at_1000
5322
+ value: 30.412
5323
+ - type: map_at_3
5324
+ value: 0.688
5325
+ - type: map_at_5
5326
+ value: 1.079
5327
+ - type: mrr_at_1
5328
+ value: 96
5329
+ - type: mrr_at_10
5330
+ value: 98
5331
+ - type: mrr_at_100
5332
+ value: 98
5333
+ - type: mrr_at_1000
5334
+ value: 98
5335
+ - type: mrr_at_3
5336
+ value: 98
5337
+ - type: mrr_at_5
5338
+ value: 98
5339
+ - type: ndcg_at_1
5340
+ value: 89
5341
+ - type: ndcg_at_10
5342
+ value: 79.646
5343
+ - type: ndcg_at_100
5344
+ value: 62.217999999999996
5345
+ - type: ndcg_at_1000
5346
+ value: 55.13400000000001
5347
+ - type: ndcg_at_3
5348
+ value: 83.458
5349
+ - type: ndcg_at_5
5350
+ value: 80.982
5351
+ - type: precision_at_1
5352
+ value: 96
5353
+ - type: precision_at_10
5354
+ value: 84.6
5355
+ - type: precision_at_100
5356
+ value: 64.34
5357
+ - type: precision_at_1000
5358
+ value: 24.534
5359
+ - type: precision_at_3
5360
+ value: 88.667
5361
+ - type: precision_at_5
5362
+ value: 85.6
5363
+ - type: recall_at_1
5364
+ value: 0.253
5365
+ - type: recall_at_10
5366
+ value: 2.253
5367
+ - type: recall_at_100
5368
+ value: 15.606
5369
+ - type: recall_at_1000
5370
+ value: 51.595
5371
+ - type: recall_at_3
5372
+ value: 0.7100000000000001
5373
+ - type: recall_at_5
5374
+ value: 1.139
5375
+ - task:
5376
+ type: Retrieval
5377
+ dataset:
5378
+ type: mteb/touche2020
5379
+ name: MTEB Touche2020
5380
+ config: default
5381
+ split: test
5382
+ revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
5383
+ metrics:
5384
+ - type: map_at_1
5385
+ value: 3.0540000000000003
5386
+ - type: map_at_10
5387
+ value: 13.078999999999999
5388
+ - type: map_at_100
5389
+ value: 19.468
5390
+ - type: map_at_1000
5391
+ value: 21.006
5392
+ - type: map_at_3
5393
+ value: 6.8629999999999995
5394
+ - type: map_at_5
5395
+ value: 9.187
5396
+ - type: mrr_at_1
5397
+ value: 42.857
5398
+ - type: mrr_at_10
5399
+ value: 56.735
5400
+ - type: mrr_at_100
5401
+ value: 57.352000000000004
5402
+ - type: mrr_at_1000
5403
+ value: 57.352000000000004
5404
+ - type: mrr_at_3
5405
+ value: 52.721
5406
+ - type: mrr_at_5
5407
+ value: 54.66
5408
+ - type: ndcg_at_1
5409
+ value: 38.775999999999996
5410
+ - type: ndcg_at_10
5411
+ value: 31.469
5412
+ - type: ndcg_at_100
5413
+ value: 42.016999999999996
5414
+ - type: ndcg_at_1000
5415
+ value: 52.60399999999999
5416
+ - type: ndcg_at_3
5417
+ value: 35.894
5418
+ - type: ndcg_at_5
5419
+ value: 33.873
5420
+ - type: precision_at_1
5421
+ value: 42.857
5422
+ - type: precision_at_10
5423
+ value: 27.346999999999998
5424
+ - type: precision_at_100
5425
+ value: 8.327
5426
+ - type: precision_at_1000
5427
+ value: 1.551
5428
+ - type: precision_at_3
5429
+ value: 36.735
5430
+ - type: precision_at_5
5431
+ value: 33.469
5432
+ - type: recall_at_1
5433
+ value: 3.0540000000000003
5434
+ - type: recall_at_10
5435
+ value: 19.185
5436
+ - type: recall_at_100
5437
+ value: 51.056000000000004
5438
+ - type: recall_at_1000
5439
+ value: 82.814
5440
+ - type: recall_at_3
5441
+ value: 7.961
5442
+ - type: recall_at_5
5443
+ value: 11.829
5444
+ - task:
5445
+ type: Classification
5446
+ dataset:
5447
+ type: mteb/toxic_conversations_50k
5448
+ name: MTEB ToxicConversationsClassification
5449
+ config: default
5450
+ split: test
5451
+ revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de
5452
+ metrics:
5453
+ - type: accuracy
5454
+ value: 64.9346
5455
+ - type: ap
5456
+ value: 12.121605736777527
5457
+ - type: f1
5458
+ value: 50.169902005887955
5459
+ - task:
5460
+ type: Classification
5461
+ dataset:
5462
+ type: mteb/tweet_sentiment_extraction
5463
+ name: MTEB TweetSentimentExtractionClassification
5464
+ config: default
5465
+ split: test
5466
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
5467
+ metrics:
5468
+ - type: accuracy
5469
+ value: 56.72608941709111
5470
+ - type: f1
5471
+ value: 57.0702928875253
5472
+ - task:
5473
+ type: Clustering
5474
+ dataset:
5475
+ type: mteb/twentynewsgroups-clustering
5476
+ name: MTEB TwentyNewsgroupsClustering
5477
+ config: default
5478
+ split: test
5479
+ revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
5480
+ metrics:
5481
+ - type: v_measure
5482
+ value: 37.72671554400943
5483
+ - task:
5484
+ type: PairClassification
5485
+ dataset:
5486
+ type: mteb/twittersemeval2015-pairclassification
5487
+ name: MTEB TwitterSemEval2015
5488
+ config: default
5489
+ split: test
5490
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
5491
+ metrics:
5492
+ - type: cos_sim_accuracy
5493
+ value: 82.84556237706384
5494
+ - type: cos_sim_ap
5495
+ value: 63.28364215788651
5496
+ - type: cos_sim_f1
5497
+ value: 60.00000000000001
5498
+ - type: cos_sim_precision
5499
+ value: 54.45161290322581
5500
+ - type: cos_sim_recall
5501
+ value: 66.80738786279683
5502
+ - type: dot_accuracy
5503
+ value: 82.84556237706384
5504
+ - type: dot_ap
5505
+ value: 63.28364302860433
5506
+ - type: dot_f1
5507
+ value: 60.00000000000001
5508
+ - type: dot_precision
5509
+ value: 54.45161290322581
5510
+ - type: dot_recall
5511
+ value: 66.80738786279683
5512
+ - type: euclidean_accuracy
5513
+ value: 82.84556237706384
5514
+ - type: euclidean_ap
5515
+ value: 63.28363625097978
5516
+ - type: euclidean_f1
5517
+ value: 60.00000000000001
5518
+ - type: euclidean_precision
5519
+ value: 54.45161290322581
5520
+ - type: euclidean_recall
5521
+ value: 66.80738786279683
5522
+ - type: manhattan_accuracy
5523
+ value: 82.86940454193241
5524
+ - type: manhattan_ap
5525
+ value: 63.244773709836764
5526
+ - type: manhattan_f1
5527
+ value: 60.12680942696495
5528
+ - type: manhattan_precision
5529
+ value: 55.00109433136353
5530
+ - type: manhattan_recall
5531
+ value: 66.3060686015831
5532
+ - type: max_accuracy
5533
+ value: 82.86940454193241
5534
+ - type: max_ap
5535
+ value: 63.28364302860433
5536
+ - type: max_f1
5537
+ value: 60.12680942696495
5538
+ - task:
5539
+ type: PairClassification
5540
+ dataset:
5541
+ type: mteb/twitterurlcorpus-pairclassification
5542
+ name: MTEB TwitterURLCorpus
5543
+ config: default
5544
+ split: test
5545
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
5546
+ metrics:
5547
+ - type: cos_sim_accuracy
5548
+ value: 88.32033220786278
5549
+ - type: cos_sim_ap
5550
+ value: 84.71928176006863
5551
+ - type: cos_sim_f1
5552
+ value: 76.51483333969684
5553
+ - type: cos_sim_precision
5554
+ value: 75.89184276300841
5555
+ - type: cos_sim_recall
5556
+ value: 77.14813674160764
5557
+ - type: dot_accuracy
5558
+ value: 88.32033220786278
5559
+ - type: dot_ap
5560
+ value: 84.71928330149228
5561
+ - type: dot_f1
5562
+ value: 76.51483333969684
5563
+ - type: dot_precision
5564
+ value: 75.89184276300841
5565
+ - type: dot_recall
5566
+ value: 77.14813674160764
5567
+ - type: euclidean_accuracy
5568
+ value: 88.32033220786278
5569
+ - type: euclidean_ap
5570
+ value: 84.71928045384345
5571
+ - type: euclidean_f1
5572
+ value: 76.51483333969684
5573
+ - type: euclidean_precision
5574
+ value: 75.89184276300841
5575
+ - type: euclidean_recall
5576
+ value: 77.14813674160764
5577
+ - type: manhattan_accuracy
5578
+ value: 88.27570147863545
5579
+ - type: manhattan_ap
5580
+ value: 84.68523541579755
5581
+ - type: manhattan_f1
5582
+ value: 76.51512269355146
5583
+ - type: manhattan_precision
5584
+ value: 75.62608107091825
5585
+ - type: manhattan_recall
5586
+ value: 77.42531567600862
5587
+ - type: max_accuracy
5588
+ value: 88.32033220786278
5589
+ - type: max_ap
5590
+ value: 84.71928330149228
5591
+ - type: max_f1
5592
+ value: 76.51512269355146
5593
+ - task:
5594
+ type: Clustering
5595
+ dataset:
5596
+ type: jinaai/cities_wiki_clustering
5597
+ name: MTEB WikiCitiesClustering
5598
+ config: default
5599
+ split: test
5600
+ revision: ddc9ee9242fa65332597f70e967ecc38b9d734fa
5601
+ metrics:
5602
+ - type: v_measure
5603
+ value: 85.30624598674467
5604
+ license: apache-2.0
5605
+ ---
5606
+ ---
5607
+ <h1 align="center">Snowflake's Artic-embed-s</h1>
5608
+ <h4 align="center">
5609
+ <p>
5610
+ <a href=#news>News</a> |
5611
+ <a href=#models>Models</a> |
5612
+ <a href=#usage>Usage</a> |
5613
+ <a href="#evaluation">Evaluation</a> |
5614
+ <a href="#contact">Contact</a> |
5615
+ <a href="#faq">FAQ</a>
5616
+ <a href="#license">License</a> |
5617
+ <a href="#acknowledgement">Acknowledgement</a>
5618
+ <p>
5619
+ </h4>
5620
+
5621
+
5622
+ ## News
5623
+
5624
+
5625
+ 04/16/2024: Release the ** Arctic-embed ** family of text empedding models. The releases are state-of-the-art for Retrieval quality at each of their representative size profiles. [Technical Report]() is coming shortly. For more details, please refer to our Github: [Arctic-Text-Embed](https://github.com/Snowflake/Arctic-Text-Embed).
5626
+
5627
+
5628
+ ## Models
5629
+
5630
+
5631
+ Arctic-Embed is a suite of text embedding models that focuses on creating high-quality retrieval models optimized for performance.
5632
+
5633
+
5634
+ The `arctic-embedding` models achieve **state-of-the-art performance on the MTEB/BEIR leaderboard** for each of their size variants. Evaluation is performed using these [scripts](https://github.com/Snowflake-Labs/arctic-embed/tree/main/src). As shown below, each class of model size achieves SOTA retrieval accuracy compared to other top models.
5635
+
5636
+
5637
+ The models are trained by leveraging existing open-source text representation models, such as bert-base-uncased, and are trained in a multi-stage pipeline to optimize their retrieval performance. First, the models are trained with large batches of query-document pairs where negatives are derived in-batch—pretraining leverages about 400m samples of a mix of public datasets and proprietary web search data. Following pretraining models are further optimized with long training on a smaller dataset (about 1m samples) of triplets of query, positive document, and negative document derived from hard harmful mining. Mining of the negatives and data curation is crucial to retrieval accuracy. A detailed technical report will be available shortly.
5638
+
5639
+
5640
+ | Name | MTEB Retrieval Score (NDCG @ 10) | Parameters (Millions) | Embedding Dimension |
5641
+ | ----------------------------------------------------------------------- | -------------------------------- | --------------------- | ------------------- |
5642
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-xs/) | 50.15 | 22 | 384 |
5643
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-s/) | 51.98 | 33 | 384 |
5644
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-m/) | 54.90 | 110 | 768 |
5645
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-m-long/) | 54.83 | 137 | 768 |
5646
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-l/) | 55.98 | 335 | 1024 |
5647
+
5648
+
5649
+ Aside from being great open-source models, the largest model, [arctic-embed-l](https://huggingface.co/Snowflake/arctic-embed-l/), can serve as a natural replacement for closed-source embedding, as shown below.
5650
+
5651
+
5652
+ | Model Name | MTEB Retrieval Score (NDCG @ 10) |
5653
+ | ------------------------------------------------------------------ | -------------------------------- |
5654
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-l/) | 55.98 |
5655
+ | Google-gecko-text-embedding | 55.7 |
5656
+ | text-embedding-3-large | 55.44 |
5657
+ | Cohere-embed-english-v3.0 | 55.00 |
5658
+ | bge-large-en-v1.5 | 54.29 |
5659
+
5660
+
5661
+ ### [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-xs/)
5662
+
5663
+
5664
+ This tiny model packs quite the punch based on the [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) model. With only 22m parameters and 384 dimensions, this model should meet even the strictest latency/TCO budgets. Despite its size, its retrieval accuracy is closer to that of models with 100m paramers.
5665
+
5666
+
5667
+ | Model Name | MTEB Retrieval Score (NDCG @ 10) |
5668
+ | ------------------------------------------------------------------- | -------------------------------- |
5669
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-xs/) | 50.15 |
5670
+ | GIST-all-MiniLM-L6-v2 | 45.12 |
5671
+ | gte-tiny | 44.92 |
5672
+ | all-MiniLM-L6-v2 | 41.95 |
5673
+ | bge-micro-v2 | 42.56 |
5674
+
5675
+
5676
+ ### Arctic-embed-m
5677
+
5678
+
5679
+ Based on the [all-MiniLM-L12-v2](https://huggingface.co/intfloat/e5-base-unsupervised) model, this small model does not trade off retrieval accuracy for its small size. With only 33m parameters and 384 dimensions, this model should easily allow scaling to large datasets.
5680
+
5681
+
5682
+ | Model Name | MTEB Retrieval Score (NDCG @ 10) |
5683
+ | ------------------------------------------------------------------ | -------------------------------- |
5684
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-s/) | 51.98 |
5685
+ | bge-small-en-v1.5 | 51.68 |
5686
+ | Cohere-embed-english-light-v3.0 | 51.34 |
5687
+ | text-embedding-3-small | 51.08 |
5688
+ | e5-small-v2 | 49.04 |
5689
+
5690
+
5691
+ ### [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-m-long/)
5692
+
5693
+
5694
+ Based on the [nomic-embed-text-v1](https://huggingface.co/nomic-ai/nomic-embed-text-v1) model, this long-context variant of our medium-sized model is perfect for workloads that can be constrained by the regular 512 token context of our other models. Without the use of RPE, this model supports up to 2048 tokens. With RPE, it can scale to 8192!
5695
+
5696
+
5697
+ | Model Name | MTEB Retrieval Score (NDCG @ 10) |
5698
+ | ------------------------------------------------------------------ | -------------------------------- |
5699
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-m/) | 54.90 |
5700
+ | bge-base-en-v1.5 | 53.25 |
5701
+ | nomic-embed-text-v1.5 | 53.01 |
5702
+ | GIST-Embedding-v0 | 52.31 |
5703
+ | gte-base | 52.31 |
5704
+
5705
+
5706
+ ### Arctic-embed-m
5707
+
5708
+
5709
+ Based on the [intfloat/e5-base-unsupervised](https://huggingface.co/intfloat/e5-base-unsupervised) model, this medium model is the workhorse that provides the best retrieval performance without slowing down inference.
5710
+
5711
+
5712
+ | Model Name | MTEB Retrieval Score (NDCG @ 10) |
5713
+ | ------------------------------------------------------------------ | -------------------------------- |
5714
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-m/) | 54.90 |
5715
+ | bge-base-en-v1.5 | 53.25 |
5716
+ | nomic-embed-text-v1.5 | 53.25 |
5717
+ | GIST-Embedding-v0 | 52.31 |
5718
+ | gte-base | 52.31 |
5719
+
5720
+
5721
+ ### [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-l/)
5722
+
5723
+
5724
+ Based on the [intfloat/e5-large-unsupervised](https://huggingface.co/intfloat/e5-large-unsupervised) model, this small model does not sacrifice retrieval accuracy for its small size.
5725
+
5726
+
5727
+ | Model Name | MTEB Retrieval Score (NDCG @ 10) |
5728
+ | ------------------------------------------------------------------ | -------------------------------- |
5729
+ | [arctic-embed-s](https://huggingface.co/Snowflake/arctic-embed-l/) | 55.98 |
5730
+ | UAE-Large-V1 | 54.66 |
5731
+ | bge-large-en-v1.5 | 54.29 |
5732
+ | mxbai-embed-large-v1 | 54.39 |
5733
+ | e5-Large-v2 | 50.56 |
5734
+
5735
+
5736
+ ## Usage
5737
+
5738
+
5739
+ ### Using Huggingface transformers
5740
+
5741
+
5742
+ You can use the transformers package to use an arctic-embed model, as shown below. For optimal retrieval quality, use the CLS token to embed each text portion and use the query prefix below (just on the query).
5743
+
5744
+
5745
+
5746
+ ```python
5747
+ import torch
5748
+ from transformers import AutoModel, AutoTokenizer
5749
+
5750
+ tokenizer = AutoTokenizer.from_pretrained('Snowflake/arctic-embed-')
5751
+ model = AutoModel.from_pretrained('Snowflake/arctic-embed-s', add_pooling_layer=False)
5752
+ model.eval()
5753
+
5754
+ query_prefix = 'Represent this sentence for searching relevant passages: '
5755
+ queries = ['what is snowflake?', 'Where can I get the best tacos?']
5756
+ queries_with_prefix = ["{}{}".format(query_prefix, i) for i in queries]
5757
+ query_tokens = tokenizer(queries_with_prefix, padding=True, truncation=True, return_tensors='pt', max_length=512)
5758
+
5759
+ documents = ['The Data Cloud!', 'Mexico City of Course!']
5760
+ document_tokens = tokenizer(documents, padding=True, truncation=True, return_tensors='pt', max_length=512)
5761
+
5762
+ # Compute token embeddings
5763
+ with torch.no_grad():
5764
+ query_embeddings = model(**query_tokens)[0][:, 0]
5765
+ doument_embeddings = model(**document_tokens)[0][:, 0]
5766
+
5767
+
5768
+ # normalize embeddings
5769
+ query_embeddings = torch.nn.functional.normalize(query_embeddings, p=2, dim=1)
5770
+ doument_embeddings = torch.nn.functional.normalize(doument_embeddings, p=2, dim=1)
5771
+
5772
+ scores = torch.mm(query_embeddings, doument_embeddings.transpose(0, 1))
5773
+ for query, query_scores in zip(queries, scores):
5774
+ doc_score_pairs = list(zip(documents, query_scores))
5775
+ doc_score_pairs = sorted(doc_score_pairs, key=lambda x: x[1], reverse=True)
5776
+ #Output passages & scores
5777
+ print("Query:", query)
5778
+ for document, score in doc_score_pairs:
5779
+ print(score, document)
5780
+ ```
5781
+
5782
+
5783
+ If you use the long context model with more than 2048 tokens, ensure that you initialize the model like below instead. This will use [RPE](https://arxiv.org/abs/2104.09864) to allow up to 8192 tokens.
5784
+
5785
+
5786
+ ``` py
5787
+ model = AutoModel.from_pretrained('Snowflake/arctic-embed-m-long', trust_remote_code=True, rotary_scaling_factor=2)
5788
+ ```
5789
+
5790
+
5791
+ ## FAQ
5792
+
5793
+
5794
+ TBD
5795
+
5796
+
5797
+ ## Contact
5798
+
5799
+
5800
+ Feel free to open an issue or pull request if you have any questions or suggestions about this project.
5801
+ You also can email Daniel Campos([email protected]).
5802
+
5803
+
5804
+ ## License
5805
+
5806
+
5807
+ Arctic is licensed under the [Apache-2](https://www.apache.org/licenses/LICENSE-2.0). The released models can be used for commercial purposes free of charge.
5808
+
5809
+
5810
+ ## Acknowledgement
5811
+
5812
+
5813
+ We want to thank the open-source community, which has provided the great building blocks upon which we could make our models.
5814
+ We thank our modeling engineers, Danmei Xu, Luke Merrick, Gaurav Nuti, and Daniel Campos, for making these great models possible.
5815
+ We thank our leadership, Himabindu Pucha, Kelvin So, Vivek Raghunathan, and Sridhar Ramaswamy, for supporting this work.
5816
+ We also thank the open-source community for producing the great models we could build on top of and making these releases possible.
5817
+ Finally, we thank the researchers who created BEIR and MTEB benchmarks.
5818
+ It is largely thanks to their tireless work to define what better looks like that we could improve model performance.