Hugging Face's logo

TencentBAC
/
Conan-embedding-v1

sentence-transformers
PyTorch
Safetensors
bert
mteb
Eval Results
Model card Files Community
8
Vurkty commited on
Commit
c292072
•
1 Parent(s): 07c86ea

First commit

Browse files
Files changed (1) hide show
  1. README.md +1076 -0
README.md CHANGED
@@ -1,3 +1,1079 @@
1
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: cc-by-nc-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ tags:
3
+ - mteb
4
+ model-index:
5
+ - name: conan-embedding
6
+ results:
7
+ - task:
8
+ type: STS
9
+ dataset:
10
+ type: C-MTEB/AFQMC
11
+ name: MTEB AFQMC
12
+ config: default
13
+ split: validation
14
+ revision: None
15
+ metrics:
16
+ - type: cos_sim_pearson
17
+ value: 57.32391831434286
18
+ - type: cos_sim_spearman
19
+ value: 60.95420518306528
20
+ - type: euclidean_pearson
21
+ value: 58.73713689471779
22
+ - type: euclidean_spearman
23
+ value: 60.05871977323687
24
+ - type: manhattan_pearson
25
+ value: 58.71439394187201
26
+ - type: manhattan_spearman
27
+ value: 60.03726849511567
28
+ - task:
29
+ type: STS
30
+ dataset:
31
+ type: C-MTEB/ATEC
32
+ name: MTEB ATEC
33
+ config: default
34
+ split: test
35
+ revision: None
36
+ metrics:
37
+ - type: cos_sim_pearson
38
+ value: 57.416459629680325
39
+ - type: cos_sim_spearman
40
+ value: 58.78935983944373
41
+ - type: euclidean_pearson
42
+ value: 62.569916488206054
43
+ - type: euclidean_spearman
44
+ value: 58.32089170859326
45
+ - type: manhattan_pearson
46
+ value: 62.552144365725816
47
+ - type: manhattan_spearman
48
+ value: 58.31304102953674
49
+ - task:
50
+ type: Classification
51
+ dataset:
52
+ type: mteb/amazon_reviews_multi
53
+ name: MTEB AmazonReviewsClassification (zh)
54
+ config: zh
55
+ split: test
56
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
57
+ metrics:
58
+ - type: accuracy
59
+ value: 50.364
60
+ - type: f1
61
+ value: 47.373487235615706
62
+ - task:
63
+ type: STS
64
+ dataset:
65
+ type: C-MTEB/BQ
66
+ name: MTEB BQ
67
+ config: default
68
+ split: test
69
+ revision: None
70
+ metrics:
71
+ - type: cos_sim_pearson
72
+ value: 73.16013393385914
73
+ - type: cos_sim_spearman
74
+ value: 74.79454418123198
75
+ - type: euclidean_pearson
76
+ value: 72.91991570850215
77
+ - type: euclidean_spearman
78
+ value: 74.40420227973465
79
+ - type: manhattan_pearson
80
+ value: 72.91482392990748
81
+ - type: manhattan_spearman
82
+ value: 74.40097720245406
83
+ - task:
84
+ type: Clustering
85
+ dataset:
86
+ type: C-MTEB/CLSClusteringP2P
87
+ name: MTEB CLSClusteringP2P
88
+ config: default
89
+ split: test
90
+ revision: None
91
+ metrics:
92
+ - type: v_measure
93
+ value: 59.86170547809245
94
+ - task:
95
+ type: Clustering
96
+ dataset:
97
+ type: C-MTEB/CLSClusteringS2S
98
+ name: MTEB CLSClusteringS2S
99
+ config: default
100
+ split: test
101
+ revision: None
102
+ metrics:
103
+ - type: v_measure
104
+ value: 50.38135526839833
105
+ - task:
106
+ type: Reranking
107
+ dataset:
108
+ type: C-MTEB/CMedQAv1-reranking
109
+ name: MTEB CMedQAv1
110
+ config: default
111
+ split: test
112
+ revision: None
113
+ metrics:
114
+ - type: map
115
+ value: 90.92142640302838
116
+ - type: mrr
117
+ value: 92.76190476190476
118
+ - task:
119
+ type: Reranking
120
+ dataset:
121
+ type: C-MTEB/CMedQAv2-reranking
122
+ name: MTEB CMedQAv2
123
+ config: default
124
+ split: test
125
+ revision: None
126
+ metrics:
127
+ - type: map
128
+ value: 90.0539331525924
129
+ - type: mrr
130
+ value: 92.00964285714285
131
+ - task:
132
+ type: Retrieval
133
+ dataset:
134
+ type: C-MTEB/CmedqaRetrieval
135
+ name: MTEB CmedqaRetrieval
136
+ config: default
137
+ split: dev
138
+ revision: None
139
+ metrics:
140
+ - type: map_at_1
141
+ value: 26.480999999999998
142
+ - type: map_at_10
143
+ value: 40.129
144
+ - type: map_at_100
145
+ value: 42.025
146
+ - type: map_at_1000
147
+ value: 42.123
148
+ - type: map_at_3
149
+ value: 35.644
150
+ - type: map_at_5
151
+ value: 38.187
152
+ - type: mrr_at_1
153
+ value: 40.01
154
+ - type: mrr_at_10
155
+ value: 48.886
156
+ - type: mrr_at_100
157
+ value: 49.825
158
+ - type: mrr_at_1000
159
+ value: 49.864000000000004
160
+ - type: mrr_at_3
161
+ value: 46.178000000000004
162
+ - type: mrr_at_5
163
+ value: 47.711999999999996
164
+ - type: ndcg_at_1
165
+ value: 40.01
166
+ - type: ndcg_at_10
167
+ value: 47.032000000000004
168
+ - type: ndcg_at_100
169
+ value: 54.135
170
+ - type: ndcg_at_1000
171
+ value: 55.821
172
+ - type: ndcg_at_3
173
+ value: 41.377
174
+ - type: ndcg_at_5
175
+ value: 43.808
176
+ - type: precision_at_1
177
+ value: 40.01
178
+ - type: precision_at_10
179
+ value: 10.495000000000001
180
+ - type: precision_at_100
181
+ value: 1.628
182
+ - type: precision_at_1000
183
+ value: 0.184
184
+ - type: precision_at_3
185
+ value: 23.648
186
+ - type: precision_at_5
187
+ value: 17.224
188
+ - type: recall_at_1
189
+ value: 26.480999999999998
190
+ - type: recall_at_10
191
+ value: 58.557
192
+ - type: recall_at_100
193
+ value: 87.52799999999999
194
+ - type: recall_at_1000
195
+ value: 98.80600000000001
196
+ - type: recall_at_3
197
+ value: 41.628
198
+ - type: recall_at_5
199
+ value: 49.013
200
+ - task:
201
+ type: PairClassification
202
+ dataset:
203
+ type: C-MTEB/CMNLI
204
+ name: MTEB Cmnli
205
+ config: default
206
+ split: validation
207
+ revision: None
208
+ metrics:
209
+ - type: cos_sim_accuracy
210
+ value: 86.5423932651834
211
+ - type: cos_sim_ap
212
+ value: 92.8551948361576
213
+ - type: cos_sim_f1
214
+ value: 87.07992733878292
215
+ - type: cos_sim_precision
216
+ value: 84.6391525049658
217
+ - type: cos_sim_recall
218
+ value: 89.66565349544074
219
+ - type: dot_accuracy
220
+ value: 77.05351773902585
221
+ - type: dot_ap
222
+ value: 85.41568844524294
223
+ - type: dot_f1
224
+ value: 79.17229905375896
225
+ - type: dot_precision
226
+ value: 71.29213483146067
227
+ - type: dot_recall
228
+ value: 89.01098901098901
229
+ - type: euclidean_accuracy
230
+ value: 84.49789536981359
231
+ - type: euclidean_ap
232
+ value: 91.54179322199462
233
+ - type: euclidean_f1
234
+ value: 85.58362369337979
235
+ - type: euclidean_precision
236
+ value: 80.08966782147951
237
+ - type: euclidean_recall
238
+ value: 91.88683656768764
239
+ - type: manhattan_accuracy
240
+ value: 84.41371016235718
241
+ - type: manhattan_ap
242
+ value: 91.5209564727476
243
+ - type: manhattan_f1
244
+ value: 85.5386606276286
245
+ - type: manhattan_precision
246
+ value: 79.38350680544436
247
+ - type: manhattan_recall
248
+ value: 92.72854804769698
249
+ - type: max_accuracy
250
+ value: 86.5423932651834
251
+ - type: max_ap
252
+ value: 92.8551948361576
253
+ - type: max_f1
254
+ value: 87.07992733878292
255
+ - task:
256
+ type: Retrieval
257
+ dataset:
258
+ type: C-MTEB/CovidRetrieval
259
+ name: MTEB CovidRetrieval
260
+ config: default
261
+ split: dev
262
+ revision: None
263
+ metrics:
264
+ - type: map_at_1
265
+ value: 82.824
266
+ - type: map_at_10
267
+ value: 89.749
268
+ - type: map_at_100
269
+ value: 89.79899999999999
270
+ - type: map_at_1000
271
+ value: 89.8
272
+ - type: map_at_3
273
+ value: 89.18599999999999
274
+ - type: map_at_5
275
+ value: 89.586
276
+ - type: mrr_at_1
277
+ value: 83.035
278
+ - type: mrr_at_10
279
+ value: 89.699
280
+ - type: mrr_at_100
281
+ value: 89.749
282
+ - type: mrr_at_1000
283
+ value: 89.749
284
+ - type: mrr_at_3
285
+ value: 89.18199999999999
286
+ - type: mrr_at_5
287
+ value: 89.582
288
+ - type: ndcg_at_1
289
+ value: 83.14
290
+ - type: ndcg_at_10
291
+ value: 92.059
292
+ - type: ndcg_at_100
293
+ value: 92.292
294
+ - type: ndcg_at_1000
295
+ value: 92.304
296
+ - type: ndcg_at_3
297
+ value: 91.033
298
+ - type: ndcg_at_5
299
+ value: 91.716
300
+ - type: precision_at_1
301
+ value: 83.14
302
+ - type: precision_at_10
303
+ value: 9.989
304
+ - type: precision_at_100
305
+ value: 1.009
306
+ - type: precision_at_1000
307
+ value: 0.101
308
+ - type: precision_at_3
309
+ value: 32.35
310
+ - type: precision_at_5
311
+ value: 19.747
312
+ - type: recall_at_1
313
+ value: 82.824
314
+ - type: recall_at_10
315
+ value: 98.84100000000001
316
+ - type: recall_at_100
317
+ value: 99.895
318
+ - type: recall_at_1000
319
+ value: 100.0
320
+ - type: recall_at_3
321
+ value: 96.207
322
+ - type: recall_at_5
323
+ value: 97.84
324
+ - task:
325
+ type: Retrieval
326
+ dataset:
327
+ type: C-MTEB/DuRetrieval
328
+ name: MTEB DuRetrieval
329
+ config: default
330
+ split: dev
331
+ revision: None
332
+ metrics:
333
+ - type: map_at_1
334
+ value: 26.839000000000002
335
+ - type: map_at_10
336
+ value: 81.363
337
+ - type: map_at_100
338
+ value: 84.265
339
+ - type: map_at_1000
340
+ value: 84.29299999999999
341
+ - type: map_at_3
342
+ value: 56.593
343
+ - type: map_at_5
344
+ value: 71.057
345
+ - type: mrr_at_1
346
+ value: 91.14999999999999
347
+ - type: mrr_at_10
348
+ value: 94.00800000000001
349
+ - type: mrr_at_100
350
+ value: 94.059
351
+ - type: mrr_at_1000
352
+ value: 94.06
353
+ - type: mrr_at_3
354
+ value: 93.692
355
+ - type: mrr_at_5
356
+ value: 93.874
357
+ - type: ndcg_at_1
358
+ value: 91.14999999999999
359
+ - type: ndcg_at_10
360
+ value: 88.584
361
+ - type: ndcg_at_100
362
+ value: 91.186
363
+ - type: ndcg_at_1000
364
+ value: 91.437
365
+ - type: ndcg_at_3
366
+ value: 87.287
367
+ - type: ndcg_at_5
368
+ value: 86.058
369
+ - type: precision_at_1
370
+ value: 91.14999999999999
371
+ - type: precision_at_10
372
+ value: 42.199999999999996
373
+ - type: precision_at_100
374
+ value: 4.845
375
+ - type: precision_at_1000
376
+ value: 0.49
377
+ - type: precision_at_3
378
+ value: 78.05
379
+ - type: precision_at_5
380
+ value: 65.53
381
+ - type: recall_at_1
382
+ value: 26.839000000000002
383
+ - type: recall_at_10
384
+ value: 89.91900000000001
385
+ - type: recall_at_100
386
+ value: 98.18900000000001
387
+ - type: recall_at_1000
388
+ value: 99.503
389
+ - type: recall_at_3
390
+ value: 58.622
391
+ - type: recall_at_5
392
+ value: 75.44
393
+ - task:
394
+ type: Retrieval
395
+ dataset:
396
+ type: C-MTEB/EcomRetrieval
397
+ name: MTEB EcomRetrieval
398
+ config: default
399
+ split: dev
400
+ revision: None
401
+ metrics:
402
+ - type: map_at_1
403
+ value: 55.300000000000004
404
+ - type: map_at_10
405
+ value: 65.53
406
+ - type: map_at_100
407
+ value: 66.084
408
+ - type: map_at_1000
409
+ value: 66.09
410
+ - type: map_at_3
411
+ value: 62.9
412
+ - type: map_at_5
413
+ value: 64.45
414
+ - type: mrr_at_1
415
+ value: 55.300000000000004
416
+ - type: mrr_at_10
417
+ value: 65.53
418
+ - type: mrr_at_100
419
+ value: 66.084
420
+ - type: mrr_at_1000
421
+ value: 66.09
422
+ - type: mrr_at_3
423
+ value: 62.9
424
+ - type: mrr_at_5
425
+ value: 64.45
426
+ - type: ndcg_at_1
427
+ value: 55.300000000000004
428
+ - type: ndcg_at_10
429
+ value: 70.743
430
+ - type: ndcg_at_100
431
+ value: 73.202
432
+ - type: ndcg_at_1000
433
+ value: 73.379
434
+ - type: ndcg_at_3
435
+ value: 65.366
436
+ - type: ndcg_at_5
437
+ value: 68.142
438
+ - type: precision_at_1
439
+ value: 55.300000000000004
440
+ - type: precision_at_10
441
+ value: 8.72
442
+ - type: precision_at_100
443
+ value: 0.9820000000000001
444
+ - type: precision_at_1000
445
+ value: 0.1
446
+ - type: precision_at_3
447
+ value: 24.166999999999998
448
+ - type: precision_at_5
449
+ value: 15.840000000000002
450
+ - type: recall_at_1
451
+ value: 55.300000000000004
452
+ - type: recall_at_10
453
+ value: 87.2
454
+ - type: recall_at_100
455
+ value: 98.2
456
+ - type: recall_at_1000
457
+ value: 99.6
458
+ - type: recall_at_3
459
+ value: 72.5
460
+ - type: recall_at_5
461
+ value: 79.2
462
+ - task:
463
+ type: Classification
464
+ dataset:
465
+ type: C-MTEB/IFlyTek-classification
466
+ name: MTEB IFlyTek
467
+ config: default
468
+ split: validation
469
+ revision: None
470
+ metrics:
471
+ - type: accuracy
472
+ value: 51.82762601000385
473
+ - type: f1
474
+ value: 39.89843169307487
475
+ - task:
476
+ type: Classification
477
+ dataset:
478
+ type: C-MTEB/JDReview-classification
479
+ name: MTEB JDReview
480
+ config: default
481
+ split: test
482
+ revision: None
483
+ metrics:
484
+ - type: accuracy
485
+ value: 89.13696060037525
486
+ - type: ap
487
+ value: 60.815127909851284
488
+ - type: f1
489
+ value: 84.4053710993565
490
+ - task:
491
+ type: STS
492
+ dataset:
493
+ type: C-MTEB/LCQMC
494
+ name: MTEB LCQMC
495
+ config: default
496
+ split: test
497
+ revision: None
498
+ metrics:
499
+ - type: cos_sim_pearson
500
+ value: 74.49480212942174
501
+ - type: cos_sim_spearman
502
+ value: 79.79417204577828
503
+ - type: euclidean_pearson
504
+ value: 79.53588770578706
505
+ - type: euclidean_spearman
506
+ value: 79.44601707954529
507
+ - type: manhattan_pearson
508
+ value: 79.52732262295254
509
+ - type: manhattan_spearman
510
+ value: 79.43565470474867
511
+ - task:
512
+ type: Reranking
513
+ dataset:
514
+ type: C-MTEB/Mmarco-reranking
515
+ name: MTEB MMarcoReranking
516
+ config: default
517
+ split: dev
518
+ revision: None
519
+ metrics:
520
+ - type: map
521
+ value: 41.45350792019148
522
+ - type: mrr
523
+ value: 41.2468253968254
524
+ - task:
525
+ type: Retrieval
526
+ dataset:
527
+ type: C-MTEB/MMarcoRetrieval
528
+ name: MTEB MMarcoRetrieval
529
+ config: default
530
+ split: dev
531
+ revision: None
532
+ metrics:
533
+ - type: map_at_1
534
+ value: 68.327
535
+ - type: map_at_10
536
+ value: 78.244
537
+ - type: map_at_100
538
+ value: 78.493
539
+ - type: map_at_1000
540
+ value: 78.498
541
+ - type: map_at_3
542
+ value: 76.305
543
+ - type: map_at_5
544
+ value: 77.549
545
+ - type: mrr_at_1
546
+ value: 70.63000000000001
547
+ - type: mrr_at_10
548
+ value: 78.78399999999999
549
+ - type: mrr_at_100
550
+ value: 79.001
551
+ - type: mrr_at_1000
552
+ value: 79.00500000000001
553
+ - type: mrr_at_3
554
+ value: 77.11099999999999
555
+ - type: mrr_at_5
556
+ value: 78.175
557
+ - type: ndcg_at_1
558
+ value: 70.63000000000001
559
+ - type: ndcg_at_10
560
+ value: 82.221
561
+ - type: ndcg_at_100
562
+ value: 83.281
563
+ - type: ndcg_at_1000
564
+ value: 83.403
565
+ - type: ndcg_at_3
566
+ value: 78.56400000000001
567
+ - type: ndcg_at_5
568
+ value: 80.65299999999999
569
+ - type: precision_at_1
570
+ value: 70.63000000000001
571
+ - type: precision_at_10
572
+ value: 9.983
573
+ - type: precision_at_100
574
+ value: 1.05
575
+ - type: precision_at_1000
576
+ value: 0.106
577
+ - type: precision_at_3
578
+ value: 29.69
579
+ - type: precision_at_5
580
+ value: 18.931
581
+ - type: recall_at_1
582
+ value: 68.327
583
+ - type: recall_at_10
584
+ value: 93.91000000000001
585
+ - type: recall_at_100
586
+ value: 98.56
587
+ - type: recall_at_1000
588
+ value: 99.508
589
+ - type: recall_at_3
590
+ value: 84.262
591
+ - type: recall_at_5
592
+ value: 89.21
593
+ - task:
594
+ type: Classification
595
+ dataset:
596
+ type: mteb/amazon_massive_intent
597
+ name: MTEB MassiveIntentClassification (zh-CN)
598
+ config: zh-CN
599
+ split: test
600
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
601
+ metrics:
602
+ - type: accuracy
603
+ value: 78.34566240753193
604
+ - type: f1
605
+ value: 74.84529699027114
606
+ - task:
607
+ type: Classification
608
+ dataset:
609
+ type: mteb/amazon_massive_scenario
610
+ name: MTEB MassiveScenarioClassification (zh-CN)
611
+ config: zh-CN
612
+ split: test
613
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
614
+ metrics:
615
+ - type: accuracy
616
+ value: 86.35171486213852
617
+ - type: f1
618
+ value: 85.47178380961844
619
+ - task:
620
+ type: Retrieval
621
+ dataset:
622
+ type: C-MTEB/MedicalRetrieval
623
+ name: MTEB MedicalRetrieval
624
+ config: default
625
+ split: dev
626
+ revision: None
627
+ metrics:
628
+ - type: map_at_1
629
+ value: 57.199999999999996
630
+ - type: map_at_10
631
+ value: 65.075
632
+ - type: map_at_100
633
+ value: 65.607
634
+ - type: map_at_1000
635
+ value: 65.63
636
+ - type: map_at_3
637
+ value: 63.0
638
+ - type: map_at_5
639
+ value: 64.145
640
+ - type: mrr_at_1
641
+ value: 57.099999999999994
642
+ - type: mrr_at_10
643
+ value: 65.024
644
+ - type: mrr_at_100
645
+ value: 65.556
646
+ - type: mrr_at_1000
647
+ value: 65.579
648
+ - type: mrr_at_3
649
+ value: 62.949999999999996
650
+ - type: mrr_at_5
651
+ value: 64.095
652
+ - type: ndcg_at_1
653
+ value: 57.199999999999996
654
+ - type: ndcg_at_10
655
+ value: 69.083
656
+ - type: ndcg_at_100
657
+ value: 71.844
658
+ - type: ndcg_at_1000
659
+ value: 72.41499999999999
660
+ - type: ndcg_at_3
661
+ value: 64.781
662
+ - type: ndcg_at_5
663
+ value: 66.842
664
+ - type: precision_at_1
665
+ value: 57.199999999999996
666
+ - type: precision_at_10
667
+ value: 8.18
668
+ - type: precision_at_100
669
+ value: 0.951
670
+ - type: precision_at_1000
671
+ value: 0.1
672
+ - type: precision_at_3
673
+ value: 23.3
674
+ - type: precision_at_5
675
+ value: 14.979999999999999
676
+ - type: recall_at_1
677
+ value: 57.199999999999996
678
+ - type: recall_at_10
679
+ value: 81.8
680
+ - type: recall_at_100
681
+ value: 95.1
682
+ - type: recall_at_1000
683
+ value: 99.5
684
+ - type: recall_at_3
685
+ value: 69.89999999999999
686
+ - type: recall_at_5
687
+ value: 74.9
688
+ - task:
689
+ type: Classification
690
+ dataset:
691
+ type: C-MTEB/MultilingualSentiment-classification
692
+ name: MTEB MultilingualSentiment
693
+ config: default
694
+ split: validation
695
+ revision: None
696
+ metrics:
697
+ - type: accuracy
698
+ value: 79.10000000000001
699
+ - type: f1
700
+ value: 78.84641270838914
701
+ - task:
702
+ type: PairClassification
703
+ dataset:
704
+ type: C-MTEB/OCNLI
705
+ name: MTEB Ocnli
706
+ config: default
707
+ split: validation
708
+ revision: None
709
+ metrics:
710
+ - type: cos_sim_accuracy
711
+ value: 85.76069301570114
712
+ - type: cos_sim_ap
713
+ value: 91.2632425363724
714
+ - type: cos_sim_f1
715
+ value: 86.8038133467135
716
+ - type: cos_sim_precision
717
+ value: 82.69598470363289
718
+ - type: cos_sim_recall
719
+ value: 91.34107708553326
720
+ - type: dot_accuracy
721
+ value: 79.48023822414727
722
+ - type: dot_ap
723
+ value: 86.96279505479045
724
+ - type: dot_f1
725
+ value: 81.3658536585366
726
+ - type: dot_precision
727
+ value: 75.61196736174071
728
+ - type: dot_recall
729
+ value: 88.0675818373812
730
+ - type: euclidean_accuracy
731
+ value: 84.0281537628587
732
+ - type: euclidean_ap
733
+ value: 88.51741173181273
734
+ - type: euclidean_f1
735
+ value: 85.51791850760924
736
+ - type: euclidean_precision
737
+ value: 79.90825688073394
738
+ - type: euclidean_recall
739
+ value: 91.97465681098205
740
+ - type: manhattan_accuracy
741
+ value: 84.08229561451002
742
+ - type: manhattan_ap
743
+ value: 88.47110130415778
744
+ - type: manhattan_f1
745
+ value: 85.5886663409868
746
+ - type: manhattan_precision
747
+ value: 79.63636363636364
748
+ - type: manhattan_recall
749
+ value: 92.5026399155227
750
+ - type: max_accuracy
751
+ value: 85.76069301570114
752
+ - type: max_ap
753
+ value: 91.2632425363724
754
+ - type: max_f1
755
+ value: 86.8038133467135
756
+ - task:
757
+ type: Classification
758
+ dataset:
759
+ type: C-MTEB/OnlineShopping-classification
760
+ name: MTEB OnlineShopping
761
+ config: default
762
+ split: test
763
+ revision: None
764
+ metrics:
765
+ - type: accuracy
766
+ value: 95.24999999999999
767
+ - type: ap
768
+ value: 93.62998298041074
769
+ - type: f1
770
+ value: 95.24017532648074
771
+ - task:
772
+ type: STS
773
+ dataset:
774
+ type: C-MTEB/PAWSX
775
+ name: MTEB PAWSX
776
+ config: default
777
+ split: test
778
+ revision: None
779
+ metrics:
780
+ - type: cos_sim_pearson
781
+ value: 42.4435049928267
782
+ - type: cos_sim_spearman
783
+ value: 48.30517824065838
784
+ - type: euclidean_pearson
785
+ value: 47.06361699313179
786
+ - type: euclidean_spearman
787
+ value: 47.82186765650415
788
+ - type: manhattan_pearson
789
+ value: 47.07696683801967
790
+ - type: manhattan_spearman
791
+ value: 47.8382411727149
792
+ - task:
793
+ type: STS
794
+ dataset:
795
+ type: C-MTEB/QBQTC
796
+ name: MTEB QBQTC
797
+ config: default
798
+ split: test
799
+ revision: None
800
+ metrics:
801
+ - type: cos_sim_pearson
802
+ value: 42.98576550573372
803
+ - type: cos_sim_spearman
804
+ value: 45.186068114717166
805
+ - type: euclidean_pearson
806
+ value: 34.35887865584346
807
+ - type: euclidean_spearman
808
+ value: 40.16452917420738
809
+ - type: manhattan_pearson
810
+ value: 34.32064302728564
811
+ - type: manhattan_spearman
812
+ value: 40.14426009784696
813
+ - task:
814
+ type: STS
815
+ dataset:
816
+ type: mteb/sts22-crosslingual-sts
817
+ name: MTEB STS22 (zh)
818
+ config: zh
819
+ split: test
820
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
821
+ metrics:
822
+ - type: cos_sim_pearson
823
+ value: 67.18376994296008
824
+ - type: cos_sim_spearman
825
+ value: 68.25525695381309
826
+ - type: euclidean_pearson
827
+ value: 65.10112612702181
828
+ - type: euclidean_spearman
829
+ value: 67.3953850420758
830
+ - type: manhattan_pearson
831
+ value: 64.96220731298227
832
+ - type: manhattan_spearman
833
+ value: 67.35995395052407
834
+ - task:
835
+ type: STS
836
+ dataset:
837
+ type: C-MTEB/STSB
838
+ name: MTEB STSB
839
+ config: default
840
+ split: test
841
+ revision: None
842
+ metrics:
843
+ - type: cos_sim_pearson
844
+ value: 80.23573559451846
845
+ - type: cos_sim_spearman
846
+ value: 81.88824265805962
847
+ - type: euclidean_pearson
848
+ value: 80.25198665320175
849
+ - type: euclidean_spearman
850
+ value: 80.78767800619003
851
+ - type: manhattan_pearson
852
+ value: 80.25183889871069
853
+ - type: manhattan_spearman
854
+ value: 80.77487518316391
855
+ - task:
856
+ type: Reranking
857
+ dataset:
858
+ type: C-MTEB/T2Reranking
859
+ name: MTEB T2Reranking
860
+ config: default
861
+ split: dev
862
+ revision: None
863
+ metrics:
864
+ - type: map
865
+ value: 69.0943636065547
866
+ - type: mrr
867
+ value: 79.81629026017988
868
+ - task:
869
+ type: Retrieval
870
+ dataset:
871
+ type: C-MTEB/T2Retrieval
872
+ name: MTEB T2Retrieval
873
+ config: default
874
+ split: dev
875
+ revision: None
876
+ metrics:
877
+ - type: map_at_1
878
+ value: 27.338
879
+ - type: map_at_10
880
+ value: 76.943
881
+ - type: map_at_100
882
+ value: 80.632
883
+ - type: map_at_1000
884
+ value: 80.695
885
+ - type: map_at_3
886
+ value: 53.946000000000005
887
+ - type: map_at_5
888
+ value: 66.399
889
+ - type: mrr_at_1
890
+ value: 89.47
891
+ - type: mrr_at_10
892
+ value: 92.31200000000001
893
+ - type: mrr_at_100
894
+ value: 92.406
895
+ - type: mrr_at_1000
896
+ value: 92.40899999999999
897
+ - type: mrr_at_3
898
+ value: 91.89
899
+ - type: mrr_at_5
900
+ value: 92.167
901
+ - type: ndcg_at_1
902
+ value: 89.47
903
+ - type: ndcg_at_10
904
+ value: 84.629
905
+ - type: ndcg_at_100
906
+ value: 88.278
907
+ - type: ndcg_at_1000
908
+ value: 88.871
909
+ - type: ndcg_at_3
910
+ value: 85.80600000000001
911
+ - type: ndcg_at_5
912
+ value: 84.531
913
+ - type: precision_at_1
914
+ value: 89.47
915
+ - type: precision_at_10
916
+ value: 42.077999999999996
917
+ - type: precision_at_100
918
+ value: 5.023
919
+ - type: precision_at_1000
920
+ value: 0.516
921
+ - type: precision_at_3
922
+ value: 75.016
923
+ - type: precision_at_5
924
+ value: 62.980000000000004
925
+ - type: recall_at_1
926
+ value: 27.338
927
+ - type: recall_at_10
928
+ value: 83.889
929
+ - type: recall_at_100
930
+ value: 95.674
931
+ - type: recall_at_1000
932
+ value: 98.65
933
+ - type: recall_at_3
934
+ value: 55.85099999999999
935
+ - type: recall_at_5
936
+ value: 70.131
937
+ - task:
938
+ type: Classification
939
+ dataset:
940
+ type: C-MTEB/TNews-classification
941
+ name: MTEB TNews
942
+ config: default
943
+ split: validation
944
+ revision: None
945
+ metrics:
946
+ - type: accuracy
947
+ value: 54.70900000000001
948
+ - type: f1
949
+ value: 52.74258250140307
950
+ - task:
951
+ type: Clustering
952
+ dataset:
953
+ type: C-MTEB/ThuNewsClusteringP2P
954
+ name: MTEB ThuNewsClusteringP2P
955
+ config: default
956
+ split: test
957
+ revision: None
958
+ metrics:
959
+ - type: v_measure
960
+ value: 77.01405081546925
961
+ - task:
962
+ type: Clustering
963
+ dataset:
964
+ type: C-MTEB/ThuNewsClusteringS2S
965
+ name: MTEB ThuNewsClusteringS2S
966
+ config: default
967
+ split: test
968
+ revision: None
969
+ metrics:
970
+ - type: v_measure
971
+ value: 71.50626450459885
972
+ - task:
973
+ type: Retrieval
974
+ dataset:
975
+ type: C-MTEB/VideoRetrieval
976
+ name: MTEB VideoRetrieval
977
+ config: default
978
+ split: dev
979
+ revision: None
980
+ metrics:
981
+ - type: map_at_1
982
+ value: 64.1
983
+ - type: map_at_10
984
+ value: 75.047
985
+ - type: map_at_100
986
+ value: 75.347
987
+ - type: map_at_1000
988
+ value: 75.348
989
+ - type: map_at_3
990
+ value: 73.333
991
+ - type: map_at_5
992
+ value: 74.313
993
+ - type: mrr_at_1
994
+ value: 64.1
995
+ - type: mrr_at_10
996
+ value: 75.047
997
+ - type: mrr_at_100
998
+ value: 75.347
999
+ - type: mrr_at_1000
1000
+ value: 75.348
1001
+ - type: mrr_at_3
1002
+ value: 73.333
1003
+ - type: mrr_at_5
1004
+ value: 74.313
1005
+ - type: ndcg_at_1
1006
+ value: 64.1
1007
+ - type: ndcg_at_10
1008
+ value: 79.814
1009
+ - type: ndcg_at_100
1010
+ value: 81.071
1011
+ - type: ndcg_at_1000
1012
+ value: 81.085
1013
+ - type: ndcg_at_3
1014
+ value: 76.307
1015
+ - type: ndcg_at_5
1016
+ value: 78.054
1017
+ - type: precision_at_1
1018
+ value: 64.1
1019
+ - type: precision_at_10
1020
+ value: 9.45
1021
+ - type: precision_at_100
1022
+ value: 0.9990000000000001
1023
+ - type: precision_at_1000
1024
+ value: 0.1
1025
+ - type: precision_at_3
1026
+ value: 28.299999999999997
1027
+ - type: precision_at_5
1028
+ value: 17.82
1029
+ - type: recall_at_1
1030
+ value: 64.1
1031
+ - type: recall_at_10
1032
+ value: 94.5
1033
+ - type: recall_at_100
1034
+ value: 99.9
1035
+ - type: recall_at_1000
1036
+ value: 100.0
1037
+ - type: recall_at_3
1038
+ value: 84.89999999999999
1039
+ - type: recall_at_5
1040
+ value: 89.1
1041
+ - task:
1042
+ type: Classification
1043
+ dataset:
1044
+ type: C-MTEB/waimai-classification
1045
+ name: MTEB Waimai
1046
+ config: default
1047
+ split: test
1048
+ revision: None
1049
+ metrics:
1050
+ - type: accuracy
1051
+ value: 89.61
1052
+ - type: ap
1053
+ value: 75.72595764105405
1054
+ - type: f1
1055
+ value: 88.23268907984898
1056
  license: cc-by-nc-4.0
1057
  ---
1058
+
1059
+ # Conan-embedding-v1
1060
+
1061
+ ## Performance
1062
+
1063
+ | Model | **Average** | **CLS** | **Clustering** | **Reranking** | **Retrieval** | **STS** | **Pair_CLS** |
1064
+ | :-------------------: | :---------: | :-------: | :------------: | :-----------: | :-----------: | :-------: | :----------: |
1065
+ | gte-Qwen2-7B-instruct | 72.05 | 75.09 | 66.06 | 68.92 | 76.03 | 65.33 | 87.48 |
1066
+ | xiaobu-embedding-v2 | 72.43 | 74.67 | 65.17 | 72.58 | 76.5 | 64.53 | 91.87 |
1067
+ | **Conan-embedding-v1** | **72.61** | 74.97 | 64.69 | 72.88 | 76.77 | 64.75 | 92.06 |
1068
+
1069
+ *More details will be available soon.*
1070
+
1071
+ ---
1072
+
1073
+ **About**
1074
+
1075
+ Created by the Tencent BAC Group. All rights reserved.
1076
+
1077
+ **License**
1078
+
1079
+ This work is licensed under a [Creative Commons Attribution-NonCommercial 4.0 International License](https://creativecommons.org/licenses/by-nc/4.0/).