metadata
base_model: sentence-transformers/all-MiniLM-L6-v2
datasets: []
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1490
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: >-
What are some examples of long-lived credentials used in ZenML for
authentication?
sentences:
- >2-
gs://zenml-core_cloudbuild ┃┃ │ gs://zenml-datasets ┃
┃ │
gs://zenml-internal-artifact-store ┃
┃ │
gs://zenml-kubeflow-artifact-store ┃
┃ │
gs://zenml-project-time-series-bucket ┃
┠───────────────────────┼─────────────────────────────────────────────────┨
┃ 🌀 kubernetes-cluster │
zenml-test-cluster ┃
┠───────────────────────┼─────────────────────────────────────────────────┨
┃ 🐳 docker-registry │
gcr.io/zenml-core ┃
┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Long-lived credentials (API keys, account keys)
This is the magic formula of authentication methods. When paired with
another ability, such as automatically generating short-lived API
tokens, or impersonating accounts or assuming roles, this is the ideal
authentication mechanism to use, particularly when using ZenML in
production and when sharing results with other members of your ZenML
team.
As a general best practice, but implemented particularly well for cloud
platforms, account passwords are never directly used as a credential
when authenticating to the cloud platform APIs. There is always a
process in place that exchanges the account/password credential for
another type of long-lived credential:
AWS uses the aws configure CLI command
GCP offers the gcloud auth application-default login CLI commands
Azure provides the az login CLI command
None of your original login information is stored on your local machine
or used to access workloads. Instead, an API key, account key or some
other form of intermediate credential is generated and stored on the
local host and used to authenticate to remote cloud service APIs.
- >-
─────────────────────────────────────────────────┨┃
VERSION │
1
┃
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨
┃ VERSION_DESCRIPTION │ Run
mlflow_training_pipeline.
┃
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨
┃ CREATED_AT │ 2023-03-01
09:09:06.899000
┃
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨
┃ UPDATED_AT │ 2023-03-01
09:09:06.899000
┃
- >-
un.py
Read more in the production guide.
CleanupMake sure you no longer need the resources before deleting them.
The instructions and commands that follow are DESTRUCTIVE.
Delete any AWS resources you no longer use to avoid additional charges.
You'll want to do the following:
aws s3 rm s3://your-bucket-name --recursive
aws s3api delete-bucket --bucket your-bucket-name
aws sagemaker delete-domain --domain-id <DOMAIN_ID>
aws ecr delete-repository --repository-name zenml-repository --force
aws iam detach-role-policy --role-name zenml-role --policy-arn
arn:aws:iam::aws:policy/AmazonS3FullAccess
aws iam detach-role-policy --role-name zenml-role --policy-arn
arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess
aws iam detach-role-policy --role-name zenml-role --policy-arn
arn:aws:iam::aws:policy/AmazonSageMakerFullAccess
aws iam delete-role --role-name zenml-role
Make sure to run these commands in the same AWS region where you created
the resources.
By running these cleanup commands, you will delete the S3 bucket,
SageMaker domain, ECR repository, and IAM role, along with their
associated policies. This will help you avoid any unnecessary charges
for resources you no longer need.
Remember to be cautious when deleting resources and ensure that you no
longer require them before running the deletion commands.
Conclusion
In this guide, we walked through the process of setting up an AWS stack
with ZenML to run your machine learning pipelines in a scalable and
production-ready environment. The key steps included:
Setting up credentials and the local environment by creating an IAM role
with the necessary permissions.
Creating a ZenML service connector to authenticate with AWS services
using the IAM role.
Configuring stack components, including an S3 artifact store, a
SageMaker Pipelines orchestrator, and an ECR container registry.
- source_sentence: >-
Can you provide more information on how to fetch the last successful run
of a pipeline in ZenML?
sentences:
- >-
Connecting remote storage
Transitioning to remote artifact storage.
In the previous chapters, we've been working with artifacts stored
locally on our machines. This setup is fine for individual experiments,
but as we move towards a collaborative and production-ready environment,
we need a solution that is more robust, shareable, and scalable. Enter
remote storage!
Remote storage allows us to store our artifacts in the cloud, which
means they're accessible from anywhere and by anyone with the right
permissions. This is essential for team collaboration and for managing
the larger datasets and models that come with production workloads.
When using a stack with remote storage, nothing changes except the fact
that the artifacts get materialized in a central and remote storage
location. This diagram explains the flow:
Provisioning and registering a remote artifact store
Out of the box, ZenML ships with many different supported artifact store
flavors. For convenience, here are some brief instructions on how to
quickly get up and running on the major cloud providers:
You will need to install and set up the AWS CLI on your machine as a
prerequisite, as covered in the AWS CLI documentation, before you
register the S3 Artifact Store.
The Amazon Web Services S3 Artifact Store flavor is provided by the S3
ZenML integration, you need to install it on your local machine to be
able to register an S3 Artifact Store and add it to your stack:
zenml integration install s3 -y
Having trouble with this command? You can use poetry or pip to install
the requirements of any ZenML integration directly. In order to obtain
the exact requirements of the AWS S3 integration you can use zenml
integration requirements s3.
The only configuration parameter mandatory for registering an S3
Artifact Store is the root path URI, which needs to point to an S3
bucket and take the form s3://bucket-name. In order to create a S3
bucket, refer to the AWS documentation.
- >-
Load a Model in code
There are a few different ways to load a ZenML Model in code:
Load the active model in a pipeline
You can also use the active model to get the model metadata, or the
associated artifacts directly as described in the starter guide:
from zenml import step, pipeline, get_step_context, pipeline, Model
@pipeline(model=Model(name="my_model"))
def my_pipeline():
...
@step
def my_step():
mv = get_step_context().model
print(mv.run_metadata["metadata_key"].value)
output = mv.get_artifact("my_dataset", "my_version")
output.run_metadata["accuracy"].value
Load any model via the Client
Alternatively, you can use the Client:
from zenml import step
from zenml.client import Client
from zenml.enums import ModelStages
@step
def model_evaluator_step()
...
try:
staging_zenml_model = Client().get_model_version(
model_name_or_id="<INSERT_MODEL_NAME>",
model_version_name_or_number_or_id=ModelStages.STAGING,
except KeyError:
staging_zenml_model = None
...
PreviousControlling Model versions
NextPromote a Model
Last updated 19 days ago
- >2-
more information.
Get the last run of a pipelineTo access the most recent run of a
pipeline, you can either use the last_run property or access it through
the runs list:
last_run = pipeline_model.last_run
If your most recent runs have failed, and you want to find the last run
that has succeeded, you can use the last_successful_run property
instead.
Get the latest run from a pipeline
Calling a pipeline executes it and then returns the response of the
freshly executed run.
run = training_pipeline()
The run that you get back is the model stored in the ZenML database at
the point of the method call. This means the pipeline run is still
initializing and no steps have been run. To get the latest state can get
a refreshed version from the client:
from zenml.client import Client
Client().get_pipeline_run(run.id) to get a refreshed version
Get a run via the client
If you already know the exact run that you want to fetch (e.g., from
looking at the dashboard), you can use the Client.get_pipeline_run()
method to fetch the run directly without having to query the pipeline
first:
from zenml.client import Client
pipeline_run =
Client().get_pipeline_run("first_pipeline-2023_06_20-16_20_13_274466")
Similar to pipelines, you can query runs by either ID, name, or name
prefix, and you can also discover runs through the Client or CLI via the
Client.list_pipeline_runs() or zenml pipeline runs list commands.
Run information
Each run has a collection of useful information which can help you
reproduce your runs. In the following, you can find a list of some of
the most useful pipeline run information, but there is much more
available. See the PipelineRunResponse definition for a comprehensive
list.
Status
The status of a pipeline run. There are five possible states:
initialized, failed, completed, running, and cached.
status = run.status
Configuration
- source_sentence: >-
How does the code in the given documentation visualize the embeddings
using t-SNE and UMAP?
sentences:
- >-
embeddings_2d[mask, 0],
embeddings_2d[mask, 1],c=[section_color_dict[section]],
label=section,
plt.title("t-SNE Visualization")
plt.legend()
plt.show()
def umap_visualization(embeddings, parent_sections):
umap_2d = umap.UMAP(n_components=2, random_state=42)
embeddings_2d = umap_2d.fit_transform(embeddings)
plt.figure(figsize=(8, 8))
for section in unique_parent_sections:
if section in section_color_dict:
mask = [section == ps for ps in parent_sections]
plt.scatter(
embeddings_2d[mask, 0],
embeddings_2d[mask, 1],
c=[section_color_dict[section]],
label=section,
plt.title("UMAP Visualization")
plt.legend()
plt.show()
In this stage, we have utilized the 'parent directory', which we had
previously stored in the vector store as an additional attribute, as a
means to color the values. This approach allows us to gain some insight
into the semantic space inherent in our data. It demonstrates that you
can visualize the embeddings and observe how similar chunks are grouped
together based on their semantic meaning and context.
So this step iterates through all the chunks and generates embeddings
representing each piece of text. These embeddings are then stored as an
artifact in the ZenML artifact store as a NumPy array. We separate this
generation from the point where we upload those embeddings to the vector
database to keep the pipeline modular and flexible; in the future we
might want to use a different vector database so we can just swap out
the upload step without having to re-generate the embeddings.
In the next section, we'll explore how to store these embeddings in a
vector database to enable fast and efficient retrieval of relevant
chunks at inference time.
Code Example
To explore the full code, visit the Complete Guide repository. The
embeddings generation step can be found here.
PreviousData ingestion and preprocessing
NextStoring embeddings in a vector database
Last updated 2 months ago
- >-
Amazon SageMaker
Executing individual steps in SageMaker.
SageMaker offers specialized compute instances to run your training jobs
and has a comprehensive UI to track and manage your models and logs.
ZenML's SageMaker step operator allows you to submit individual steps to
be run on Sagemaker compute instances.
When to use it
You should use the SageMaker step operator if:
one or more steps of your pipeline require computing resources (CPU,
GPU, memory) that are not provided by your orchestrator.
you have access to SageMaker. If you're using a different cloud
provider, take a look at the Vertex or AzureML step operators.
How to deploy it
Create a role in the IAM console that you want the jobs running in
SageMaker to assume. This role should at least have the
AmazonS3FullAccess and AmazonSageMakerFullAccess policies applied. Check
here for a guide on how to set up this role.
Infrastructure Deployment
A Sagemaker step operator can be deployed directly from the ZenML CLI:
zenml orchestrator deploy sagemaker_step_operator --flavor=sagemaker
--provider=aws ...
You can pass other configurations specific to the stack components as
key-value arguments. If you don't provide a name, a random one is
generated for you. For more information about how to work use the CLI
for this, please refer to the dedicated documentation section.
How to use it
To use the SageMaker step operator, we need:
The ZenML aws integration installed. If you haven't done so,
runCopyzenml integration install aws
Docker installed and running.
An IAM role with the correct permissions. See the deployment section for
detailed instructions.
An AWS container registry as part of our stack. Take a look here for a
guide on how to set that up.
- >-
Deploy with Docker
Deploying ZenML in a Docker container.
The ZenML server container image is available at
zenmldocker/zenml-server and can be used to deploy ZenML with a
container management or orchestration tool like Docker and
docker-compose, or a serverless platform like Cloud Run, Container Apps,
and more! This guide walks you through the various configuration options
that the ZenML server container expects as well as a few deployment use
cases.
Try it out locally first
If you're just looking for a quick way to deploy the ZenML server using
a container, without going through the hassle of interacting with a
container management tool like Docker and manually configuring your
container, you can use the ZenML CLI to do so. You only need to have
Docker installed and running on your machine:
zenml up --docker
This command deploys a ZenML server locally in a Docker container, then
connects your client to it. Similar to running plain zenml up, the
server and the local ZenML client share the same SQLite database.
The rest of this guide is addressed to advanced users who are looking to
manually deploy and manage a containerized ZenML server.
ZenML server configuration options
If you're planning on deploying a custom containerized ZenML server
yourself, you probably need to configure some settings for it like the
database it should use, the default user details, and more. The ZenML
server container image uses sensible defaults, so you can simply start a
container without worrying too much about the configuration. However, if
you're looking to connect the ZenML server to an external MySQL database
or secrets management service, to persist the internal SQLite database,
or simply want to control other settings like the default account, you
can do so by customizing the container's environment variables.
The following environment variables can be passed to the container:
- source_sentence: What is the purpose of the `enable_step_logs` parameter in ZenML?
sentences:
- >-
Collect information from your SQL database serviceUsing an external
MySQL-compatible database service is optional, but is recommended for
production deployments. If omitted, ZenML will default to using an
embedded SQLite database, which has the following limitations:
the SQLite database is not persisted, meaning that it will be lost if
the ZenML server pod is restarted or deleted
the SQLite database does not scale horizontally, meaning that you will
not be able to use more than one replica at a time for the ZenML server
pod
If you decide to use an external MySQL-compatible database service, you
will need to collect and prepare the following information for the Helm
chart configuration:
the hostname and port where the SQL database is reachable from the
Kubernetes cluster
the username and password that will be used to connect to the database.
It is recommended that you create a dedicated database user for the
ZenML server and that you restrict its privileges to only access the
database that will be used by ZenML. Enforcing secure SSL connections
for the user/database is also recommended. See the MySQL documentation
for more information on how to set up users and privileges.
the name of the database that will be used by ZenML. The database does
not have to exist prior to the deployment ( ZenML will create it on the
first start). However, you need to create the database if you follow the
best practice of restricting database user privileges to only access it.
if you plan on using SSL to secure the client database connection, you
may also need to prepare additional SSL certificates and keys:the TLS CA
certificate that was used to sign the server TLS certificate, if you're
using a self-signed certificate or signed by a custom certificate
authority that is not already trusted by default by most operating
systems.the TLS client certificate and key. This is only needed if you
decide to use client certificates for your DB connection (some managed
DB services support this, CloudSQL is an example).
- >2-
│ SHARED │ OWNER │ EXPIRES IN │ LABELS ┃┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨
┃ │ gcp-multi │ 9d953320-3560-4a78-817c-926a3898064d
│ 🔵 gcp │ 🔵 gcp-generic │ <multiple> │ ➖ │
default │ │ ┃
┃ │ │
│ │ 📦 gcs-bucket │ │
│ │ │ ┃
┃ │ │
│ │ 🌀 kubernetes-cluster │ │
│ │ │ ┃
┃ │ │
│ │ 🐳 docker-registry │ │
│ │ │ ┃
┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨
┃ │ gcs-multi │ ff9c0723-7451-46b7-93ef-fcf3efde30fa
│ 🔵 gcp │ 📦 gcs-bucket │ <multiple> │ ➖ │
default │ │ ┃
┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨
┃ │ gcs-langchain-slackbot │ cf3953e9-414c-4875-ba00-24c62a0dc0c5
│ 🔵 gcp │ 📦 gcs-bucket │ gs://langchain-slackbot │ ➖ │
default │ │ ┃
┗━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━┛
Local and remote availability
- >-
onfiguration, if specified overrides for this
stepenable_artifact_metadata: True
enable_artifact_visualization: True
enable_cache: False
enable_step_logs: True
step
extra: {}
step
model: {}
step
settings:
docker: {}
resources: {}
step_operator.sagemaker:
estimator_args:
instance_type: m7g.medium
Deep-dive
enable_XXX parameters
These are boolean flags for various configurations:
enable_artifact_metadata: Whether to associate metadata with artifacts
or not.
enable_artifact_visualization: Whether to attach visualizations of
artifacts.
enable_cache: Utilize caching or not.
enable_step_logs: Enable tracking step logs.
enable_artifact_metadata: True
enable_artifact_visualization: True
enable_cache: True
enable_step_logs: True
build ID
The UUID of the build to use for this pipeline. If specified, Docker
image building is skipped for remote orchestrators, and the Docker image
specified in this build is used.
build: <INSERT-BUILD-ID-HERE>
Configuring the model
Specifies the ZenML Model to use for this pipeline.
model:
name: "ModelName"
version: "production"
description: An example model
tags: ["classifier"]
Pipeline and step parameters
A dictionary of JSON-serializable parameters specified at the pipeline
or step level. For example:
parameters:
gamma: 0.01
steps:
trainer:
parameters:
gamma: 0.001
Corresponds to:
from zenml import step, pipeline
@step
def trainer(gamma: float):
print(gamma)
@pipeline
def my_pipeline(gamma: float):
print(0.01)
trainer(gamma=gamma)
- source_sentence: >-
Can I use ZenML to register an S3 Artifact Store after setting up the IAM
user and generating the access key?
sentences:
- >2-
to install and configure the AWS CLI on your hostyou don't need to care about enabling your other stack components (orchestrators, step operators, and model deployers) to have access to the artifact store through IAM roles and policies
you can combine the S3 artifact store with other stack components that
are not running in AWS
Note: When you create the IAM user for your AWS access key, please
remember to grant the created IAM user permissions to read and write to
your S3 bucket (i.e. at a minimum: s3:PutObject, s3:GetObject,
s3:ListBucket, s3:DeleteObject)
After having set up the IAM user and generated the access key, as
described in the AWS documentation, you can register the S3 Artifact
Store as follows:
zenml secret create s3_secret \
--aws_access_key_id='<YOUR_S3_ACCESS_KEY_ID>' \
--aws_secret_access_key='<YOUR_S3_SECRET_KEY>'
zenml artifact-store register s3_store -f s3 \
--path='s3://your-bucket' \
--authentication_secret=s3_secret
zenml stack register custom_stack -a s3_store ... --set
Advanced Configuration
The S3 Artifact Store accepts a range of advanced configuration options
that can be used to further customize how ZenML connects to the S3
storage service that you are using. These are accessible via the
client_kwargs, config_kwargs and s3_additional_kwargs configuration
attributes and are passed transparently to the underlying S3Fs library:
client_kwargs: arguments that will be transparently passed to the
botocore client . You can use it to configure parameters like
endpoint_url and region_name when connecting to an S3-compatible
endpoint (e.g. Minio).
config_kwargs: advanced parameters passed to botocore.client.Config.
s3_additional_kwargs: advanced parameters that are used when calling S3
API, typically used for things like ServerSideEncryption and ACL.
- >-
Evidently
How to keep your data quality in check and guard against data and model
drift with Evidently profiling
The Evidently Data Validator flavor provided with the ZenML integration
uses Evidently to perform data quality, data drift, model drift and
model performance analyses, to generate reports and run checks. The
reports and check results can be used to implement automated corrective
actions in your pipelines or to render interactive representations for
further visual interpretation, evaluation and documentation.
When would you want to use it?
Evidently is an open-source library that you can use to monitor and
debug machine learning models by analyzing the data that they use
through a powerful set of data profiling and visualization features, or
to run a variety of data and model validation reports and tests, from
data integrity tests that work with a single dataset to model evaluation
tests to data drift analysis and model performance comparison tests. All
this can be done with minimal configuration input from the user, or
customized with specialized conditions that the validation tests should
perform.
Evidently currently works with tabular data in pandas.DataFrame or CSV
file formats and can handle both regression and classification tasks.
You should use the Evidently Data Validator when you need the following
data and/or model validation features that are possible with Evidently:
Data Quality reports and tests: provides detailed feature statistics and
a feature behavior overview for a single dataset. It can also compare
any two datasets. E.g. you can use it to compare train and test data,
reference and current data, or two subgroups of one dataset.
Data Drift reports and tests: helps detects and explore feature
distribution changes in the input data by comparing two datasets with
identical schema.
- >2-
us know!
Configuration at pipeline or step levelWhen running your ZenML pipeline
with the Sagemaker orchestrator, the configuration set when configuring
the orchestrator as a ZenML component will be used by default. However,
it is possible to provide additional configuration at the pipeline or
step level. This allows you to run whole pipelines or individual steps
with alternative configurations. For example, this allows you to run the
training process with a heavier, GPU-enabled instance type, while
running other steps with lighter instances.
Additional configuration for the Sagemaker orchestrator can be passed
via SagemakerOrchestratorSettings. Here, it is possible to configure
processor_args, which is a dictionary of arguments for the Processor.
For available arguments, see the Sagemaker documentation . Currently, it
is not possible to provide custom configuration for the following
attributes:
image_uri
instance_count
sagemaker_session
entrypoint
base_job_name
env
For example, settings can be provided in the following way:
sagemaker_orchestrator_settings = SagemakerOrchestratorSettings(
processor_args={
"instance_type": "ml.t3.medium",
"volume_size_in_gb": 30
They can then be applied to a step as follows:
@step(settings={"orchestrator.sagemaker":
sagemaker_orchestrator_settings})
For example, if your ZenML component is configured to use ml.c5.xlarge
with 400GB additional storage by default, all steps will use it except
for the step above, which will use ml.t3.medium with 30GB additional
storage.
Check out this docs page for more information on how to specify settings
in general.
For more information and a full list of configurable attributes of the
Sagemaker orchestrator, check out the SDK Docs .
S3 data access in ZenML steps
model-index:
- name: zenml/finetuned-all-MiniLM-L6-v2
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 384
type: dim_384
metrics:
- type: cosine_accuracy@1
value: 0.3313253012048193
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.6024096385542169
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6927710843373494
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7710843373493976
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.3313253012048193
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2008032128514056
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13855421686746985
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07710843373493974
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.3313253012048193
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.6024096385542169
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6927710843373494
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7710843373493976
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.551850042031417
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.4812392426850258
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.48905601510986996
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.3253012048192771
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5903614457831325
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6867469879518072
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7710843373493976
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.3253012048192771
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.19678714859437746
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13734939759036144
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07710843373493974
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.3253012048192771
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.5903614457831325
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6867469879518072
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7710843373493976
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5441628856415894
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.4714883342895392
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.4790082728748276
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.3313253012048193
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5120481927710844
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6144578313253012
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.6987951807228916
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.3313253012048193
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.1706827309236948
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.12289156626506023
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.06987951807228915
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.3313253012048193
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.5120481927710844
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6144578313253012
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.6987951807228916
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5108893388836802
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.45126936316695354
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.4605530012141939
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.2891566265060241
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.4759036144578313
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.5542168674698795
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.6265060240963856
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.2891566265060241
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.15863453815261044
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.1108433734939759
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.06265060240963854
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.2891566265060241
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.4759036144578313
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.5542168674698795
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.6265060240963856
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.45650145038804574
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.40227337923121054
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.4137670603435629
name: Cosine Map@100
Then you can load this model and run inference.
