Update README.md
Browse files
README.md
CHANGED
@@ -193,7 +193,7 @@ python evals/eval_hf_datasets_v1.py \
|
|
193 |
```
|
194 |
|
195 |
## Detailed Performance
|
196 |
-
|
197 |
|
198 |
Within both these scenarios, the models were benchmarked against three different tasks:
|
199 |
|
|
|
193 |
```
|
194 |
|
195 |
## Detailed Performance
|
196 |
+
Our benchmarking process was divided into two distinct regimes, each using different datasets of ecommerce product listings: marqo-ecommerce-hard and marqo-ecommerce-easy. Both datasets contained product images and text and only differed in size. The "easy" dataset is approximately 10-30 times smaller (200k vs 4M products), and designed to accommodate rate-limited models, specifically Cohere-Embeddings-v3 and GCP-Vertex (with limits of 0.66 rps and 2 rps respectively). The "hard" dataset represents the true challenge, since it contains four million ecommerce product listings and is more representative of real-world ecommerce search scenarios.
|
197 |
|
198 |
Within both these scenarios, the models were benchmarked against three different tasks:
|
199 |
|