--- base_model: sentence-transformers/all-MiniLM-L6-v2 datasets: [] language: - en library_name: sentence-transformers license: apache-2.0 metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:1490 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: - source_sentence: What are some examples of long-lived credentials used in ZenML for authentication? sentences: - ' gs://zenml-core_cloudbuild ┃┃ │ gs://zenml-datasets ┃ ┃ │ gs://zenml-internal-artifact-store ┃ ┃ │ gs://zenml-kubeflow-artifact-store ┃ ┃ │ gs://zenml-project-time-series-bucket ┃ ┠───────────────────────┼─────────────────────────────────────────────────┨ ┃ 🌀 kubernetes-cluster │ zenml-test-cluster ┃ ┠───────────────────────┼─────────────────────────────────────────────────┨ ┃ 🐳 docker-registry │ gcr.io/zenml-core ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ Long-lived credentials (API keys, account keys) This is the magic formula of authentication methods. When paired with another ability, such as automatically generating short-lived API tokens, or impersonating accounts or assuming roles, this is the ideal authentication mechanism to use, particularly when using ZenML in production and when sharing results with other members of your ZenML team. As a general best practice, but implemented particularly well for cloud platforms, account passwords are never directly used as a credential when authenticating to the cloud platform APIs. There is always a process in place that exchanges the account/password credential for another type of long-lived credential: AWS uses the aws configure CLI command GCP offers the gcloud auth application-default login CLI commands Azure provides the az login CLI command None of your original login information is stored on your local machine or used to access workloads. Instead, an API key, account key or some other form of intermediate credential is generated and stored on the local host and used to authenticate to remote cloud service APIs.' - '─────────────────────────────────────────────────┨┃ VERSION │ 1 ┃ ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨ ┃ VERSION_DESCRIPTION │ Run #1 of the mlflow_training_pipeline. ┃ ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨ ┃ CREATED_AT │ 2023-03-01 09:09:06.899000 ┃ ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨ ┃ UPDATED_AT │ 2023-03-01 09:09:06.899000 ┃' - 'un.py Read more in the production guide. CleanupMake sure you no longer need the resources before deleting them. The instructions and commands that follow are DESTRUCTIVE. Delete any AWS resources you no longer use to avoid additional charges. You''ll want to do the following: # delete the S3 bucket aws s3 rm s3://your-bucket-name --recursive aws s3api delete-bucket --bucket your-bucket-name # delete the SageMaker domain aws sagemaker delete-domain --domain-id # delete the ECR repository aws ecr delete-repository --repository-name zenml-repository --force # detach policies from the IAM role aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonSageMakerFullAccess # delete the IAM role aws iam delete-role --role-name zenml-role Make sure to run these commands in the same AWS region where you created the resources. By running these cleanup commands, you will delete the S3 bucket, SageMaker domain, ECR repository, and IAM role, along with their associated policies. This will help you avoid any unnecessary charges for resources you no longer need. Remember to be cautious when deleting resources and ensure that you no longer require them before running the deletion commands. Conclusion In this guide, we walked through the process of setting up an AWS stack with ZenML to run your machine learning pipelines in a scalable and production-ready environment. The key steps included: Setting up credentials and the local environment by creating an IAM role with the necessary permissions. Creating a ZenML service connector to authenticate with AWS services using the IAM role. Configuring stack components, including an S3 artifact store, a SageMaker Pipelines orchestrator, and an ECR container registry.' - source_sentence: Can you provide more information on how to fetch the last successful run of a pipeline in ZenML? sentences: - 'Connecting remote storage Transitioning to remote artifact storage. In the previous chapters, we''ve been working with artifacts stored locally on our machines. This setup is fine for individual experiments, but as we move towards a collaborative and production-ready environment, we need a solution that is more robust, shareable, and scalable. Enter remote storage! Remote storage allows us to store our artifacts in the cloud, which means they''re accessible from anywhere and by anyone with the right permissions. This is essential for team collaboration and for managing the larger datasets and models that come with production workloads. When using a stack with remote storage, nothing changes except the fact that the artifacts get materialized in a central and remote storage location. This diagram explains the flow: Provisioning and registering a remote artifact store Out of the box, ZenML ships with many different supported artifact store flavors. For convenience, here are some brief instructions on how to quickly get up and running on the major cloud providers: You will need to install and set up the AWS CLI on your machine as a prerequisite, as covered in the AWS CLI documentation, before you register the S3 Artifact Store. The Amazon Web Services S3 Artifact Store flavor is provided by the S3 ZenML integration, you need to install it on your local machine to be able to register an S3 Artifact Store and add it to your stack: zenml integration install s3 -y Having trouble with this command? You can use poetry or pip to install the requirements of any ZenML integration directly. In order to obtain the exact requirements of the AWS S3 integration you can use zenml integration requirements s3. The only configuration parameter mandatory for registering an S3 Artifact Store is the root path URI, which needs to point to an S3 bucket and take the form s3://bucket-name. In order to create a S3 bucket, refer to the AWS documentation.' - 'Load a Model in code There are a few different ways to load a ZenML Model in code: Load the active model in a pipeline You can also use the active model to get the model metadata, or the associated artifacts directly as described in the starter guide: from zenml import step, pipeline, get_step_context, pipeline, Model @pipeline(model=Model(name="my_model")) def my_pipeline(): ... @step def my_step(): # Get model from active step context mv = get_step_context().model # Get metadata print(mv.run_metadata["metadata_key"].value) # Directly fetch an artifact that is attached to the model output = mv.get_artifact("my_dataset", "my_version") output.run_metadata["accuracy"].value Load any model via the Client Alternatively, you can use the Client: from zenml import step from zenml.client import Client from zenml.enums import ModelStages @step def model_evaluator_step() ... # Get staging model version try: staging_zenml_model = Client().get_model_version( model_name_or_id="", model_version_name_or_number_or_id=ModelStages.STAGING, except KeyError: staging_zenml_model = None ... PreviousControlling Model versions NextPromote a Model Last updated 19 days ago' - ' more information. Get the last run of a pipelineTo access the most recent run of a pipeline, you can either use the last_run property or access it through the runs list: last_run = pipeline_model.last_run # OR: pipeline_model.runs[0] If your most recent runs have failed, and you want to find the last run that has succeeded, you can use the last_successful_run property instead. Get the latest run from a pipeline Calling a pipeline executes it and then returns the response of the freshly executed run. run = training_pipeline() The run that you get back is the model stored in the ZenML database at the point of the method call. This means the pipeline run is still initializing and no steps have been run. To get the latest state can get a refreshed version from the client: from zenml.client import Client Client().get_pipeline_run(run.id) to get a refreshed version Get a run via the client If you already know the exact run that you want to fetch (e.g., from looking at the dashboard), you can use the Client.get_pipeline_run() method to fetch the run directly without having to query the pipeline first: from zenml.client import Client pipeline_run = Client().get_pipeline_run("first_pipeline-2023_06_20-16_20_13_274466") Similar to pipelines, you can query runs by either ID, name, or name prefix, and you can also discover runs through the Client or CLI via the Client.list_pipeline_runs() or zenml pipeline runs list commands. Run information Each run has a collection of useful information which can help you reproduce your runs. In the following, you can find a list of some of the most useful pipeline run information, but there is much more available. See the PipelineRunResponse definition for a comprehensive list. Status The status of a pipeline run. There are five possible states: initialized, failed, completed, running, and cached. status = run.status Configuration' - source_sentence: How does the code in the given documentation visualize the embeddings using t-SNE and UMAP? sentences: - ' embeddings_2d[mask, 0], embeddings_2d[mask, 1],c=[section_color_dict[section]], label=section, plt.title("t-SNE Visualization") plt.legend() plt.show() # Dimensionality reduction using UMAP def umap_visualization(embeddings, parent_sections): umap_2d = umap.UMAP(n_components=2, random_state=42) embeddings_2d = umap_2d.fit_transform(embeddings) plt.figure(figsize=(8, 8)) for section in unique_parent_sections: if section in section_color_dict: mask = [section == ps for ps in parent_sections] plt.scatter( embeddings_2d[mask, 0], embeddings_2d[mask, 1], c=[section_color_dict[section]], label=section, plt.title("UMAP Visualization") plt.legend() plt.show() In this stage, we have utilized the ''parent directory'', which we had previously stored in the vector store as an additional attribute, as a means to color the values. This approach allows us to gain some insight into the semantic space inherent in our data. It demonstrates that you can visualize the embeddings and observe how similar chunks are grouped together based on their semantic meaning and context. So this step iterates through all the chunks and generates embeddings representing each piece of text. These embeddings are then stored as an artifact in the ZenML artifact store as a NumPy array. We separate this generation from the point where we upload those embeddings to the vector database to keep the pipeline modular and flexible; in the future we might want to use a different vector database so we can just swap out the upload step without having to re-generate the embeddings. In the next section, we''ll explore how to store these embeddings in a vector database to enable fast and efficient retrieval of relevant chunks at inference time. Code Example To explore the full code, visit the Complete Guide repository. The embeddings generation step can be found here. PreviousData ingestion and preprocessing NextStoring embeddings in a vector database Last updated 2 months ago' - 'Amazon SageMaker Executing individual steps in SageMaker. SageMaker offers specialized compute instances to run your training jobs and has a comprehensive UI to track and manage your models and logs. ZenML''s SageMaker step operator allows you to submit individual steps to be run on Sagemaker compute instances. When to use it You should use the SageMaker step operator if: one or more steps of your pipeline require computing resources (CPU, GPU, memory) that are not provided by your orchestrator. you have access to SageMaker. If you''re using a different cloud provider, take a look at the Vertex or AzureML step operators. How to deploy it Create a role in the IAM console that you want the jobs running in SageMaker to assume. This role should at least have the AmazonS3FullAccess and AmazonSageMakerFullAccess policies applied. Check here for a guide on how to set up this role. Infrastructure Deployment A Sagemaker step operator can be deployed directly from the ZenML CLI: zenml orchestrator deploy sagemaker_step_operator --flavor=sagemaker --provider=aws ... You can pass other configurations specific to the stack components as key-value arguments. If you don''t provide a name, a random one is generated for you. For more information about how to work use the CLI for this, please refer to the dedicated documentation section. How to use it To use the SageMaker step operator, we need: The ZenML aws integration installed. If you haven''t done so, runCopyzenml integration install aws Docker installed and running. An IAM role with the correct permissions. See the deployment section for detailed instructions. An AWS container registry as part of our stack. Take a look here for a guide on how to set that up.' - 'Deploy with Docker Deploying ZenML in a Docker container. The ZenML server container image is available at zenmldocker/zenml-server and can be used to deploy ZenML with a container management or orchestration tool like Docker and docker-compose, or a serverless platform like Cloud Run, Container Apps, and more! This guide walks you through the various configuration options that the ZenML server container expects as well as a few deployment use cases. Try it out locally first If you''re just looking for a quick way to deploy the ZenML server using a container, without going through the hassle of interacting with a container management tool like Docker and manually configuring your container, you can use the ZenML CLI to do so. You only need to have Docker installed and running on your machine: zenml up --docker This command deploys a ZenML server locally in a Docker container, then connects your client to it. Similar to running plain zenml up, the server and the local ZenML client share the same SQLite database. The rest of this guide is addressed to advanced users who are looking to manually deploy and manage a containerized ZenML server. ZenML server configuration options If you''re planning on deploying a custom containerized ZenML server yourself, you probably need to configure some settings for it like the database it should use, the default user details, and more. The ZenML server container image uses sensible defaults, so you can simply start a container without worrying too much about the configuration. However, if you''re looking to connect the ZenML server to an external MySQL database or secrets management service, to persist the internal SQLite database, or simply want to control other settings like the default account, you can do so by customizing the container''s environment variables. The following environment variables can be passed to the container:' - source_sentence: What is the purpose of the `enable_step_logs` parameter in ZenML? sentences: - 'Collect information from your SQL database serviceUsing an external MySQL-compatible database service is optional, but is recommended for production deployments. If omitted, ZenML will default to using an embedded SQLite database, which has the following limitations: the SQLite database is not persisted, meaning that it will be lost if the ZenML server pod is restarted or deleted the SQLite database does not scale horizontally, meaning that you will not be able to use more than one replica at a time for the ZenML server pod If you decide to use an external MySQL-compatible database service, you will need to collect and prepare the following information for the Helm chart configuration: the hostname and port where the SQL database is reachable from the Kubernetes cluster the username and password that will be used to connect to the database. It is recommended that you create a dedicated database user for the ZenML server and that you restrict its privileges to only access the database that will be used by ZenML. Enforcing secure SSL connections for the user/database is also recommended. See the MySQL documentation for more information on how to set up users and privileges. the name of the database that will be used by ZenML. The database does not have to exist prior to the deployment ( ZenML will create it on the first start). However, you need to create the database if you follow the best practice of restricting database user privileges to only access it. if you plan on using SSL to secure the client database connection, you may also need to prepare additional SSL certificates and keys:the TLS CA certificate that was used to sign the server TLS certificate, if you''re using a self-signed certificate or signed by a custom certificate authority that is not already trusted by default by most operating systems.the TLS client certificate and key. This is only needed if you decide to use client certificates for your DB connection (some managed DB services support this, CloudSQL is an example).' - ' │ SHARED │ OWNER │ EXPIRES IN │ LABELS ┃┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨ ┃ │ gcp-multi │ 9d953320-3560-4a78-817c-926a3898064d │ 🔵 gcp │ 🔵 gcp-generic │ │ ➖ │ default │ │ ┃ ┃ │ │ │ │ 📦 gcs-bucket │ │ │ │ │ ┃ ┃ │ │ │ │ 🌀 kubernetes-cluster │ │ │ │ │ ┃ ┃ │ │ │ │ 🐳 docker-registry │ │ │ │ │ ┃ ┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨ ┃ │ gcs-multi │ ff9c0723-7451-46b7-93ef-fcf3efde30fa │ 🔵 gcp │ 📦 gcs-bucket │ │ ➖ │ default │ │ ┃ ┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨ ┃ │ gcs-langchain-slackbot │ cf3953e9-414c-4875-ba00-24c62a0dc0c5 │ 🔵 gcp │ 📦 gcs-bucket │ gs://langchain-slackbot │ ➖ │ default │ │ ┃ ┗━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━┛ Local and remote availability' - 'onfiguration, if specified overrides for this stepenable_artifact_metadata: True enable_artifact_visualization: True enable_cache: False enable_step_logs: True # Same as pipeline level configuration, if specified overrides for this step extra: {} # Same as pipeline level configuration, if specified overrides for this step model: {} # Same as pipeline level configuration, if specified overrides for this step settings: docker: {} resources: {} # Stack component specific settings step_operator.sagemaker: estimator_args: instance_type: m7g.medium Deep-dive enable_XXX parameters These are boolean flags for various configurations: enable_artifact_metadata: Whether to associate metadata with artifacts or not. enable_artifact_visualization: Whether to attach visualizations of artifacts. enable_cache: Utilize caching or not. enable_step_logs: Enable tracking step logs. enable_artifact_metadata: True enable_artifact_visualization: True enable_cache: True enable_step_logs: True build ID The UUID of the build to use for this pipeline. If specified, Docker image building is skipped for remote orchestrators, and the Docker image specified in this build is used. build: Configuring the model Specifies the ZenML Model to use for this pipeline. model: name: "ModelName" version: "production" description: An example model tags: ["classifier"] Pipeline and step parameters A dictionary of JSON-serializable parameters specified at the pipeline or step level. For example: parameters: gamma: 0.01 steps: trainer: parameters: gamma: 0.001 Corresponds to: from zenml import step, pipeline @step def trainer(gamma: float): # Use gamma as normal print(gamma) @pipeline def my_pipeline(gamma: float): # use gamma or pass it into the step print(0.01) trainer(gamma=gamma)' - source_sentence: Can I use ZenML to register an S3 Artifact Store after setting up the IAM user and generating the access key? sentences: - ' to install and configure the AWS CLI on your hostyou don''t need to care about enabling your other stack components (orchestrators, step operators, and model deployers) to have access to the artifact store through IAM roles and policies you can combine the S3 artifact store with other stack components that are not running in AWS Note: When you create the IAM user for your AWS access key, please remember to grant the created IAM user permissions to read and write to your S3 bucket (i.e. at a minimum: s3:PutObject, s3:GetObject, s3:ListBucket, s3:DeleteObject) After having set up the IAM user and generated the access key, as described in the AWS documentation, you can register the S3 Artifact Store as follows: # Store the AWS access key in a ZenML secret zenml secret create s3_secret \ --aws_access_key_id='''' \ --aws_secret_access_key='''' # Register the S3 artifact-store and reference the ZenML secret zenml artifact-store register s3_store -f s3 \ --path=''s3://your-bucket'' \ --authentication_secret=s3_secret # Register and set a stack with the new artifact store zenml stack register custom_stack -a s3_store ... --set Advanced Configuration The S3 Artifact Store accepts a range of advanced configuration options that can be used to further customize how ZenML connects to the S3 storage service that you are using. These are accessible via the client_kwargs, config_kwargs and s3_additional_kwargs configuration attributes and are passed transparently to the underlying S3Fs library: client_kwargs: arguments that will be transparently passed to the botocore client . You can use it to configure parameters like endpoint_url and region_name when connecting to an S3-compatible endpoint (e.g. Minio). config_kwargs: advanced parameters passed to botocore.client.Config. s3_additional_kwargs: advanced parameters that are used when calling S3 API, typically used for things like ServerSideEncryption and ACL.' - 'Evidently How to keep your data quality in check and guard against data and model drift with Evidently profiling The Evidently Data Validator flavor provided with the ZenML integration uses Evidently to perform data quality, data drift, model drift and model performance analyses, to generate reports and run checks. The reports and check results can be used to implement automated corrective actions in your pipelines or to render interactive representations for further visual interpretation, evaluation and documentation. When would you want to use it? Evidently is an open-source library that you can use to monitor and debug machine learning models by analyzing the data that they use through a powerful set of data profiling and visualization features, or to run a variety of data and model validation reports and tests, from data integrity tests that work with a single dataset to model evaluation tests to data drift analysis and model performance comparison tests. All this can be done with minimal configuration input from the user, or customized with specialized conditions that the validation tests should perform. Evidently currently works with tabular data in pandas.DataFrame or CSV file formats and can handle both regression and classification tasks. You should use the Evidently Data Validator when you need the following data and/or model validation features that are possible with Evidently: Data Quality reports and tests: provides detailed feature statistics and a feature behavior overview for a single dataset. It can also compare any two datasets. E.g. you can use it to compare train and test data, reference and current data, or two subgroups of one dataset. Data Drift reports and tests: helps detects and explore feature distribution changes in the input data by comparing two datasets with identical schema.' - ' us know! Configuration at pipeline or step levelWhen running your ZenML pipeline with the Sagemaker orchestrator, the configuration set when configuring the orchestrator as a ZenML component will be used by default. However, it is possible to provide additional configuration at the pipeline or step level. This allows you to run whole pipelines or individual steps with alternative configurations. For example, this allows you to run the training process with a heavier, GPU-enabled instance type, while running other steps with lighter instances. Additional configuration for the Sagemaker orchestrator can be passed via SagemakerOrchestratorSettings. Here, it is possible to configure processor_args, which is a dictionary of arguments for the Processor. For available arguments, see the Sagemaker documentation . Currently, it is not possible to provide custom configuration for the following attributes: image_uri instance_count sagemaker_session entrypoint base_job_name env For example, settings can be provided in the following way: sagemaker_orchestrator_settings = SagemakerOrchestratorSettings( processor_args={ "instance_type": "ml.t3.medium", "volume_size_in_gb": 30 They can then be applied to a step as follows: @step(settings={"orchestrator.sagemaker": sagemaker_orchestrator_settings}) For example, if your ZenML component is configured to use ml.c5.xlarge with 400GB additional storage by default, all steps will use it except for the step above, which will use ml.t3.medium with 30GB additional storage. Check out this docs page for more information on how to specify settings in general. For more information and a full list of configurable attributes of the Sagemaker orchestrator, check out the SDK Docs . S3 data access in ZenML steps' model-index: - name: zenml/finetuned-all-MiniLM-L6-v2 results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 384 type: dim_384 metrics: - type: cosine_accuracy@1 value: 0.3313253012048193 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.6024096385542169 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6927710843373494 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7710843373493976 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.3313253012048193 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2008032128514056 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.13855421686746985 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.07710843373493974 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.3313253012048193 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.6024096385542169 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6927710843373494 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7710843373493976 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.551850042031417 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4812392426850258 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.48905601510986996 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 0.3253012048192771 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5903614457831325 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6867469879518072 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7710843373493976 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.3253012048192771 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.19678714859437746 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.13734939759036144 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.07710843373493974 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.3253012048192771 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.5903614457831325 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6867469879518072 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7710843373493976 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5441628856415894 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4714883342895392 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4790082728748276 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 0.3313253012048193 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5120481927710844 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6144578313253012 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.6987951807228916 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.3313253012048193 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.1706827309236948 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.12289156626506023 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.06987951807228915 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.3313253012048193 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.5120481927710844 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6144578313253012 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.6987951807228916 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5108893388836802 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.45126936316695354 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4605530012141939 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.2891566265060241 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.4759036144578313 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.5542168674698795 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.6265060240963856 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.2891566265060241 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.15863453815261044 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.1108433734939759 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.06265060240963854 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.2891566265060241 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.4759036144578313 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.5542168674698795 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.6265060240963856 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.45650145038804574 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.40227337923121054 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4137670603435629 name: Cosine Map@100 --- # zenml/finetuned-all-MiniLM-L6-v2 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) - **Maximum Sequence Length:** 256 tokens - **Output Dimensionality:** 384 tokens - **Similarity Function:** Cosine Similarity - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("zenml/finetuned-all-MiniLM-L6-v2") # Run inference sentences = [ 'Can I use ZenML to register an S3 Artifact Store after setting up the IAM user and generating the access key?', " to install and configure the AWS CLI on your hostyou don't need to care about enabling your other stack components (orchestrators, step operators, and model deployers) to have access to the artifact store through IAM roles and policies\n\nyou can combine the S3 artifact store with other stack components that are not running in AWS\n\nNote: When you create the IAM user for your AWS access key, please remember to grant the created IAM user permissions to read and write to your S3 bucket (i.e. at a minimum: s3:PutObject, s3:GetObject, s3:ListBucket, s3:DeleteObject)\n\nAfter having set up the IAM user and generated the access key, as described in the AWS documentation, you can register the S3 Artifact Store as follows:\n\n# Store the AWS access key in a ZenML secret\n\nzenml secret create s3_secret \\\n\n--aws_access_key_id='' \\\n\n--aws_secret_access_key=''\n\n# Register the S3 artifact-store and reference the ZenML secret\n\nzenml artifact-store register s3_store -f s3 \\\n\n--path='s3://your-bucket' \\\n\n--authentication_secret=s3_secret\n\n# Register and set a stack with the new artifact store\n\nzenml stack register custom_stack -a s3_store ... --set\n\nAdvanced Configuration\n\nThe S3 Artifact Store accepts a range of advanced configuration options that can be used to further customize how ZenML connects to the S3 storage service that you are using. These are accessible via the client_kwargs, config_kwargs and s3_additional_kwargs configuration attributes and are passed transparently to the underlying S3Fs library:\n\nclient_kwargs: arguments that will be transparently passed to the botocore client . You can use it to configure parameters like endpoint_url and region_name when connecting to an S3-compatible endpoint (e.g. Minio).\n\nconfig_kwargs: advanced parameters passed to botocore.client.Config.\n\ns3_additional_kwargs: advanced parameters that are used when calling S3 API, typically used for things like ServerSideEncryption and ACL.", ' us know!\n\nConfiguration at pipeline or step levelWhen running your ZenML pipeline with the Sagemaker orchestrator, the configuration set when configuring the orchestrator as a ZenML component will be used by default. However, it is possible to provide additional configuration at the pipeline or step level. This allows you to run whole pipelines or individual steps with alternative configurations. For example, this allows you to run the training process with a heavier, GPU-enabled instance type, while running other steps with lighter instances.\n\nAdditional configuration for the Sagemaker orchestrator can be passed via SagemakerOrchestratorSettings. Here, it is possible to configure processor_args, which is a dictionary of arguments for the Processor. For available arguments, see the Sagemaker documentation . Currently, it is not possible to provide custom configuration for the following attributes:\n\nimage_uri\n\ninstance_count\n\nsagemaker_session\n\nentrypoint\n\nbase_job_name\n\nenv\n\nFor example, settings can be provided in the following way:\n\nsagemaker_orchestrator_settings = SagemakerOrchestratorSettings(\n\nprocessor_args={\n\n"instance_type": "ml.t3.medium",\n\n"volume_size_in_gb": 30\n\nThey can then be applied to a step as follows:\n\n@step(settings={"orchestrator.sagemaker": sagemaker_orchestrator_settings})\n\nFor example, if your ZenML component is configured to use ml.c5.xlarge with 400GB additional storage by default, all steps will use it except for the step above, which will use ml.t3.medium with 30GB additional storage.\n\nCheck out this docs page for more information on how to specify settings in general.\n\nFor more information and a full list of configurable attributes of the Sagemaker orchestrator, check out the SDK Docs .\n\nS3 data access in ZenML steps', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_384` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.3313 | | cosine_accuracy@3 | 0.6024 | | cosine_accuracy@5 | 0.6928 | | cosine_accuracy@10 | 0.7711 | | cosine_precision@1 | 0.3313 | | cosine_precision@3 | 0.2008 | | cosine_precision@5 | 0.1386 | | cosine_precision@10 | 0.0771 | | cosine_recall@1 | 0.3313 | | cosine_recall@3 | 0.6024 | | cosine_recall@5 | 0.6928 | | cosine_recall@10 | 0.7711 | | cosine_ndcg@10 | 0.5519 | | cosine_mrr@10 | 0.4812 | | **cosine_map@100** | **0.4891** | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:----------| | cosine_accuracy@1 | 0.3253 | | cosine_accuracy@3 | 0.5904 | | cosine_accuracy@5 | 0.6867 | | cosine_accuracy@10 | 0.7711 | | cosine_precision@1 | 0.3253 | | cosine_precision@3 | 0.1968 | | cosine_precision@5 | 0.1373 | | cosine_precision@10 | 0.0771 | | cosine_recall@1 | 0.3253 | | cosine_recall@3 | 0.5904 | | cosine_recall@5 | 0.6867 | | cosine_recall@10 | 0.7711 | | cosine_ndcg@10 | 0.5442 | | cosine_mrr@10 | 0.4715 | | **cosine_map@100** | **0.479** | #### Information Retrieval * Dataset: `dim_128` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.3313 | | cosine_accuracy@3 | 0.512 | | cosine_accuracy@5 | 0.6145 | | cosine_accuracy@10 | 0.6988 | | cosine_precision@1 | 0.3313 | | cosine_precision@3 | 0.1707 | | cosine_precision@5 | 0.1229 | | cosine_precision@10 | 0.0699 | | cosine_recall@1 | 0.3313 | | cosine_recall@3 | 0.512 | | cosine_recall@5 | 0.6145 | | cosine_recall@10 | 0.6988 | | cosine_ndcg@10 | 0.5109 | | cosine_mrr@10 | 0.4513 | | **cosine_map@100** | **0.4606** | #### Information Retrieval * Dataset: `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.2892 | | cosine_accuracy@3 | 0.4759 | | cosine_accuracy@5 | 0.5542 | | cosine_accuracy@10 | 0.6265 | | cosine_precision@1 | 0.2892 | | cosine_precision@3 | 0.1586 | | cosine_precision@5 | 0.1108 | | cosine_precision@10 | 0.0627 | | cosine_recall@1 | 0.2892 | | cosine_recall@3 | 0.4759 | | cosine_recall@5 | 0.5542 | | cosine_recall@10 | 0.6265 | | cosine_ndcg@10 | 0.4565 | | cosine_mrr@10 | 0.4023 | | **cosine_map@100** | **0.4138** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 1,490 training samples * Columns: positive and anchor * Approximate statistics based on the first 1000 samples: | | positive | anchor | |:--------|:----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details |
  • min: 9 tokens
  • mean: 21.16 tokens
  • max: 49 tokens
|
  • min: 21 tokens
  • mean: 239.82 tokens
  • max: 256 tokens
| * Samples: | positive | anchor | |:------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Could you explain how to configure step-specific settings like the experiment_tracker and step_operator in ZenML docs? | docs.

failure_hook_source and success_hook_sourceThe source of the failure and success hooks can be specified.

Step-specific configuration

A lot of pipeline-level configuration can also be applied at a step level (as we have already seen with the enable_cache flag). However, there is some configuration that is step-specific, meaning it cannot be applied at a pipeline level, but only at a step level.

experiment_tracker: Name of the experiment_tracker to enable for this step. This experiment_tracker should be defined in the active stack with the same name.

step_operator: Name of the step_operator to enable for this step. This step_operator should be defined in the active stack with the same name.

outputs: This is configuration of the output artifacts of this step. This is further keyed by output name (by default, step outputs are named output). The most interesting configuration here is the materializer_source, which is the UDF path of the materializer in code to use for this output (e.g. materializers.some_data.materializer.materializer_class). Read more about this source path here.

PreviousHow to configure a pipeline with a YAML

NextRuntime settings for Docker, resources, and stack components

Last updated 19 days ago | | How do I configure ZenML to use the Azure Key Vault as a secrets store backend? | COUNT_NAME>@.iam.gserviceaccount.comUsing the Azure Key Vault as a secrets store backend

The Azure Secrets Store uses the ZenML Azure Service Connector under the hood to authenticate with the Azure Key Vault API. This means that you can use any of the authentication methods supported by the Azure Service Connector to authenticate with the Azure Key Vault API.

Example configuration for the Azure Key Vault Secrets Store:

zenml:

# ...

# Secrets store settings. This is used to store centralized secrets.

secretsStore:

# Set to false to disable the secrets store.

enabled: true

# The type of the secrets store

type: azure

# Configuration for the Azure Key Vault secrets store

azure:

# The name of the Azure Key Vault. This must be set to point to the Azure

# Key Vault instance that you want to use.

key_vault_name:

# The Azure Service Connector authentication method to use.

authMethod: service-principal

# The Azure Service Connector configuration.

authConfig:

# The Azure application service principal credentials to use to

# authenticate with the Azure Key Vault API.

client_id:

client_secret:

tenant_id:

Using the HashiCorp Vault as a secrets store backend

To use the HashiCorp Vault service as a Secrets Store back-end, it must be configured in the Helm values:

zenml:

# ...

# Secrets store settings. This is used to store centralized secrets.

secretsStore:

# Set to false to disable the secrets store.

enabled: true

# The type of the secrets store

type: hashicorp

# Configuration for the HashiCorp Vault secrets store

hashicorp:

# The url of the HashiCorp Vault server to use

vault_addr: https://vault.example.com

# The token used to authenticate with the Vault server

vault_token:

# The Vault Enterprise namespace. Not required for Vault OSS.

vault_namespace:

Using a custom secrets store backend implementation | | How do I register a service connector with AWS using ZenML? | r the local cloud provider CLI (AWS in this case):AWS_PROFILE=connectors zenml service-connector register aws-sts-token --type aws --auto-configure --auth-method sts-token

Example Command Output

⠸ Registering service connector 'aws-sts-token'...

Successfully registered service connector `aws-sts-token` with access to the following resources:

┏━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ RESOURCE TYPE │ RESOURCE NAMES ┃

┠───────────────────────┼──────────────────────────────────────────────┨

┃ 🔶 aws-generic │ us-east-1 ┃

┠───────────────────────┼──────────────────────────────────────────────┨

┃ 📦 s3-bucket │ s3://zenfiles ┃

┃ │ s3://zenml-demos ┃

┃ │ s3://zenml-generative-chat ┃

┃ │ s3://zenml-public-datasets ┃

┠───────────────────────┼──────────────────────────────────────────────┨

┃ 🌀 kubernetes-cluster │ zenhacks-cluster ┃

┠───────────────────────┼──────────────────────────────────────────────┨

┃ 🐳 docker-registry │ 715803424590.dkr.ecr.us-east-1.amazonaws.com ┃

┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

The Service Connector is now configured with a short-lived token that will expire after some time. You can verify this by inspecting the Service Connector:

zenml service-connector describe aws-sts-token

Example Command Output

Service connector 'aws-sts-token' of type 'aws' with id '63e14350-6719-4255-b3f5-0539c8f7c303' is owned by user 'default' and is 'private'.

'aws-sts-token' aws Service Connector Details

┏━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ PROPERTY │ VALUE ┃ | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `tf32`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: True - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_384_cosine_map@100 | dim_64_cosine_map@100 | |:----------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | 0.6667 | 1 | 0.4231 | 0.4458 | 0.4525 | 0.3907 | | 2.0 | 3 | 0.4526 | 0.4745 | 0.4875 | 0.4114 | | **2.6667** | **4** | **0.4606** | **0.479** | **0.4891** | **0.4138** | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.14 - Sentence Transformers: 3.0.1 - Transformers: 4.41.2 - PyTorch: 2.3.1+cu121 - Accelerate: 0.31.0 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```