README.md

---
base_model: sentence-transformers/all-MiniLM-L6-v2
datasets: []
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1490
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: What are some examples of long-lived credentials used in ZenML
    for authentication?
  sentences:
  - ' gs://zenml-core_cloudbuild                      ┃┃                       │ gs://zenml-datasets                             ┃


    ┃                       │ gs://zenml-internal-artifact-store              ┃


    ┃                       │ gs://zenml-kubeflow-artifact-store              ┃


    ┃                       │ gs://zenml-project-time-series-bucket           ┃


    ┠───────────────────────┼─────────────────────────────────────────────────┨


    ┃ 🌀 kubernetes-cluster │ zenml-test-cluster                              ┃


    ┠───────────────────────┼─────────────────────────────────────────────────┨


    ┃  🐳 docker-registry   │ gcr.io/zenml-core                               ┃


    ┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛


    Long-lived credentials (API keys, account keys)


    This is the magic formula of authentication methods. When paired with another
    ability, such as automatically generating short-lived API tokens, or impersonating
    accounts or assuming roles, this is the ideal authentication mechanism to use,
    particularly when using ZenML in production and when sharing results with other
    members of your ZenML team.


    As a general best practice, but implemented particularly well for cloud platforms,
    account passwords are never directly used as a credential when authenticating
    to the cloud platform APIs. There is always a process in place that exchanges
    the account/password credential for another type of long-lived credential:


    AWS uses the aws configure CLI command


    GCP offers the gcloud auth application-default login CLI commands


    Azure provides the az login CLI command


    None of your original login information is stored on your local machine or used
    to access workloads. Instead, an API key, account key or some other form of intermediate
    credential is generated and stored on the local host and used to authenticate
    to remote cloud service APIs.'
  - '─────────────────────────────────────────────────┨┃ VERSION                │
    1                                                                                                                                                                                                                                              ┃


    ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨


    ┃ VERSION_DESCRIPTION    │ Run #1 of the mlflow_training_pipeline.                                                                                                                                                                                                        ┃


    ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨


    ┃ CREATED_AT             │ 2023-03-01 09:09:06.899000                                                                                                                                                                                                                     ┃


    ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨


    ┃ UPDATED_AT             │ 2023-03-01 09:09:06.899000                                                                                                                                                                                                                     ┃'
  - 'un.py


    Read more in the production guide.


    CleanupMake sure you no longer need the resources before deleting them. The instructions
    and commands that follow are DESTRUCTIVE.


    Delete any AWS resources you no longer use to avoid additional charges. You''ll
    want to do the following:


    # delete the S3 bucket


    aws s3 rm s3://your-bucket-name --recursive


    aws s3api delete-bucket --bucket your-bucket-name


    # delete the SageMaker domain


    aws sagemaker delete-domain --domain-id <DOMAIN_ID>


    # delete the ECR repository


    aws ecr delete-repository --repository-name zenml-repository --force


    # detach policies from the IAM role


    aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess


    aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess


    aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonSageMakerFullAccess


    # delete the IAM role


    aws iam delete-role --role-name zenml-role


    Make sure to run these commands in the same AWS region where you created the resources.


    By running these cleanup commands, you will delete the S3 bucket, SageMaker domain,
    ECR repository, and IAM role, along with their associated policies. This will
    help you avoid any unnecessary charges for resources you no longer need.


    Remember to be cautious when deleting resources and ensure that you no longer
    require them before running the deletion commands.


    Conclusion


    In this guide, we walked through the process of setting up an AWS stack with ZenML
    to run your machine learning pipelines in a scalable and production-ready environment.
    The key steps included:


    Setting up credentials and the local environment by creating an IAM role with
    the necessary permissions.


    Creating a ZenML service connector to authenticate with AWS services using the
    IAM role.


    Configuring stack components, including an S3 artifact store, a SageMaker Pipelines
    orchestrator, and an ECR container registry.'
- source_sentence: Can you provide more information on how to fetch the last successful
    run of a pipeline in ZenML?
  sentences:
  - 'Connecting remote storage


    Transitioning to remote artifact storage.


    In the previous chapters, we''ve been working with artifacts stored locally on
    our machines. This setup is fine for individual experiments, but as we move towards
    a collaborative and production-ready environment, we need a solution that is more
    robust, shareable, and scalable. Enter remote storage!


    Remote storage allows us to store our artifacts in the cloud, which means they''re
    accessible from anywhere and by anyone with the right permissions. This is essential
    for team collaboration and for managing the larger datasets and models that come
    with production workloads.


    When using a stack with remote storage, nothing changes except the fact that the
    artifacts get materialized in a central and remote storage location. This diagram
    explains the flow:


    Provisioning and registering a remote artifact store


    Out of the box, ZenML ships with many different supported artifact store flavors.
    For convenience, here are some brief instructions on how to quickly get up and
    running on the major cloud providers:


    You will need to install and set up the AWS CLI on your machine as a prerequisite,
    as covered in the AWS CLI documentation, before you register the S3 Artifact Store.


    The Amazon Web Services S3 Artifact Store flavor is provided by the S3 ZenML integration,
    you need to install it on your local machine to be able to register an S3 Artifact
    Store and add it to your stack:


    zenml integration install s3 -y


    Having trouble with this command? You can use poetry or pip to install the requirements
    of any ZenML integration directly. In order to obtain the exact requirements of
    the AWS S3 integration you can use zenml integration requirements s3.


    The only configuration parameter mandatory for registering an S3 Artifact Store
    is the root path URI, which needs to point to an S3 bucket and take the form s3://bucket-name.
    In order to create a S3 bucket, refer to the AWS documentation.'
  - 'Load a Model in code


    There are a few different ways to load a ZenML Model in code:


    Load the active model in a pipeline


    You can also use the active model to get the model metadata, or the associated
    artifacts directly as described in the starter guide:


    from zenml import step, pipeline, get_step_context, pipeline, Model


    @pipeline(model=Model(name="my_model"))


    def my_pipeline():


    ...


    @step


    def my_step():


    # Get model from active step context


    mv = get_step_context().model


    # Get metadata


    print(mv.run_metadata["metadata_key"].value)


    # Directly fetch an artifact that is attached to the model


    output = mv.get_artifact("my_dataset", "my_version")


    output.run_metadata["accuracy"].value


    Load any model via the Client


    Alternatively, you can use the Client:


    from zenml import step


    from zenml.client import Client


    from zenml.enums import ModelStages


    @step


    def model_evaluator_step()


    ...


    # Get staging model version


    try:


    staging_zenml_model = Client().get_model_version(


    model_name_or_id="<INSERT_MODEL_NAME>",


    model_version_name_or_number_or_id=ModelStages.STAGING,


    except KeyError:


    staging_zenml_model = None


    ...


    PreviousControlling Model versions


    NextPromote a Model


    Last updated 19 days ago'
  - ' more information.


    Get the last run of a pipelineTo access the most recent run of a pipeline, you
    can either use the last_run property or access it through the runs list:


    last_run = pipeline_model.last_run  # OR: pipeline_model.runs[0]


    If your most recent runs have failed, and you want to find the last run that has
    succeeded, you can use the last_successful_run property instead.


    Get the latest run from a pipeline


    Calling a pipeline executes it and then returns the response of the freshly executed
    run.


    run = training_pipeline()


    The run that you get back is the model stored in the ZenML database at the point
    of the method call. This means the pipeline run is still initializing and no steps
    have been run. To get the latest state can get a refreshed version from the client:


    from zenml.client import Client


    Client().get_pipeline_run(run.id) to get a refreshed version


    Get a run via the client


    If you already know the exact run that you want to fetch (e.g., from looking at
    the dashboard), you can use the Client.get_pipeline_run() method to fetch the
    run directly without having to query the pipeline first:


    from zenml.client import Client


    pipeline_run = Client().get_pipeline_run("first_pipeline-2023_06_20-16_20_13_274466")


    Similar to pipelines, you can query runs by either ID, name, or name prefix, and
    you can also discover runs through the Client or CLI via the Client.list_pipeline_runs()
    or zenml pipeline runs list commands.


    Run information


    Each run has a collection of useful information which can help you reproduce your
    runs. In the following, you can find a list of some of the most useful pipeline
    run information, but there is much more available. See the PipelineRunResponse
    definition for a comprehensive list.


    Status


    The status of a pipeline run. There are five possible states: initialized, failed,
    completed, running, and cached.


    status = run.status


    Configuration'
- source_sentence: How does the code in the given documentation visualize the embeddings
    using t-SNE and UMAP?
  sentences:
  - '


    embeddings_2d[mask, 0],


    embeddings_2d[mask, 1],c=[section_color_dict[section]],


    label=section,


    plt.title("t-SNE Visualization")


    plt.legend()


    plt.show()


    # Dimensionality reduction using UMAP


    def umap_visualization(embeddings, parent_sections):


    umap_2d = umap.UMAP(n_components=2, random_state=42)


    embeddings_2d = umap_2d.fit_transform(embeddings)


    plt.figure(figsize=(8, 8))


    for section in unique_parent_sections:


    if section in section_color_dict:


    mask = [section == ps for ps in parent_sections]


    plt.scatter(


    embeddings_2d[mask, 0],


    embeddings_2d[mask, 1],


    c=[section_color_dict[section]],


    label=section,


    plt.title("UMAP Visualization")


    plt.legend()


    plt.show()


    In this stage, we have utilized the ''parent directory'', which we had previously
    stored in the vector store as an additional attribute, as a means to color the
    values. This approach allows us to gain some insight into the semantic space inherent
    in our data. It demonstrates that you can visualize the embeddings and observe
    how similar chunks are grouped together based on their semantic meaning and context.


    So this step iterates through all the chunks and generates embeddings representing
    each piece of text. These embeddings are then stored as an artifact in the ZenML
    artifact store as a NumPy array. We separate this generation from the point where
    we upload those embeddings to the vector database to keep the pipeline modular
    and flexible; in the future we might want to use a different vector database so
    we can just swap out the upload step without having to re-generate the embeddings.


    In the next section, we''ll explore how to store these embeddings in a vector
    database to enable fast and efficient retrieval of relevant chunks at inference
    time.


    Code Example


    To explore the full code, visit the Complete Guide repository. The embeddings
    generation step can be found here.


    PreviousData ingestion and preprocessing


    NextStoring embeddings in a vector database


    Last updated 2 months ago'
  - 'Amazon SageMaker


    Executing individual steps in SageMaker.


    SageMaker offers specialized compute instances to run your training jobs and has
    a comprehensive UI to track and manage your models and logs. ZenML''s SageMaker
    step operator allows you to submit individual steps to be run on Sagemaker compute
    instances.


    When to use it


    You should use the SageMaker step operator if:


    one or more steps of your pipeline require computing resources (CPU, GPU, memory)
    that are not provided by your orchestrator.


    you have access to SageMaker. If you''re using a different cloud provider, take
    a look at the Vertex or AzureML step operators.


    How to deploy it


    Create a role in the IAM console that you want the jobs running in SageMaker to
    assume. This role should at least have the AmazonS3FullAccess and AmazonSageMakerFullAccess
    policies applied. Check here for a guide on how to set up this role.


    Infrastructure Deployment


    A Sagemaker step operator can be deployed directly from the ZenML CLI:


    zenml orchestrator deploy sagemaker_step_operator --flavor=sagemaker --provider=aws
    ...


    You can pass other configurations specific to the stack components as key-value
    arguments. If you don''t provide a name, a random one is generated for you. For
    more information about how to work use the CLI for this, please refer to the dedicated
    documentation section.


    How to use it


    To use the SageMaker step operator, we need:


    The ZenML aws integration installed. If you haven''t done so, runCopyzenml integration
    install aws


    Docker installed and running.


    An IAM role with the correct permissions. See the deployment section for detailed
    instructions.


    An AWS container registry as part of our stack. Take a look here for a guide on
    how to set that up.'
  - 'Deploy with Docker


    Deploying ZenML in a Docker container.


    The ZenML server container image is available at zenmldocker/zenml-server and
    can be used to deploy ZenML with a container management or orchestration tool
    like Docker and docker-compose, or a serverless platform like Cloud Run, Container
    Apps, and more! This guide walks you through the various configuration options
    that the ZenML server container expects as well as a few deployment use cases.


    Try it out locally first


    If you''re just looking for a quick way to deploy the ZenML server using a container,
    without going through the hassle of interacting with a container management tool
    like Docker and manually configuring your container, you can use the ZenML CLI
    to do so. You only need to have Docker installed and running on your machine:


    zenml up --docker


    This command deploys a ZenML server locally in a Docker container, then connects
    your client to it. Similar to running plain zenml up, the server and the local
    ZenML client share the same SQLite database.


    The rest of this guide is addressed to advanced users who are looking to manually
    deploy and manage a containerized ZenML server.


    ZenML server configuration options


    If you''re planning on deploying a custom containerized ZenML server yourself,
    you probably need to configure some settings for it like the database it should
    use, the default user details, and more. The ZenML server container image uses
    sensible defaults, so you can simply start a container without worrying too much
    about the configuration. However, if you''re looking to connect the ZenML server
    to an external MySQL database or secrets management service, to persist the internal
    SQLite database, or simply want to control other settings like the default account,
    you can do so by customizing the container''s environment variables.


    The following environment variables can be passed to the container:'
- source_sentence: What is the purpose of the `enable_step_logs` parameter in ZenML?
  sentences:
  - 'Collect information from your SQL database serviceUsing an external MySQL-compatible
    database service is optional, but is recommended for production deployments. If
    omitted, ZenML will default to using an embedded SQLite database, which has the
    following limitations:


    the SQLite database is not persisted, meaning that it will be lost if the ZenML
    server pod is restarted or deleted


    the SQLite database does not scale horizontally, meaning that you will not be
    able to use more than one replica at a time for the ZenML server pod


    If you decide to use an external MySQL-compatible database service, you will need
    to collect and prepare the following information for the Helm chart configuration:


    the hostname and port where the SQL database is reachable from the Kubernetes
    cluster


    the username and password that will be used to connect to the database. It is
    recommended that you create a dedicated database user for the ZenML server and
    that you restrict its privileges to only access the database that will be used
    by ZenML. Enforcing secure SSL connections for the user/database is also recommended.
    See the MySQL documentation for more information on how to set up users and privileges.


    the name of the database that will be used by ZenML. The database does not have
    to exist prior to the deployment ( ZenML will create it on the first start). However,
    you need to create the database if you follow the best practice of restricting
    database user privileges to only access it.


    if you plan on using SSL to secure the client database connection, you may also
    need to prepare additional SSL certificates and keys:the TLS CA certificate that
    was used to sign the server TLS certificate, if you''re using a self-signed certificate
    or signed by a custom certificate authority that is not already trusted by default
    by most operating systems.the TLS client certificate and key. This is only needed
    if you decide to use client certificates for your DB connection (some managed
    DB services support this, CloudSQL is an example).'
  - '        │ SHARED │ OWNER   │ EXPIRES IN │ LABELS ┃┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨


    ┃        │ gcp-multi              │ 9d953320-3560-4a78-817c-926a3898064d │ 🔵 gcp
    │ 🔵 gcp-generic        │ <multiple>              │ ➖     │ default │            │        ┃


    ┃        │                        │                                      │        │
    📦 gcs-bucket         │                         │        │         │            │        ┃


    ┃        │                        │                                      │        │
    🌀 kubernetes-cluster │                         │        │         │            │        ┃


    ┃        │                        │                                      │        │
    🐳 docker-registry    │                         │        │         │            │        ┃


    ┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨


    ┃        │ gcs-multi              │ ff9c0723-7451-46b7-93ef-fcf3efde30fa │ 🔵 gcp
    │ 📦 gcs-bucket         │ <multiple>              │ ➖     │ default │            │        ┃


    ┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨


    ┃        │ gcs-langchain-slackbot │ cf3953e9-414c-4875-ba00-24c62a0dc0c5 │ 🔵 gcp
    │ 📦 gcs-bucket         │ gs://langchain-slackbot │ ➖     │ default │            │        ┃


    ┗━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━┛


    Local and remote availability'
  - 'onfiguration, if specified overrides for this stepenable_artifact_metadata: True


    enable_artifact_visualization: True


    enable_cache: False


    enable_step_logs: True


    # Same as pipeline level configuration, if specified overrides for this step


    extra: {}


    # Same as pipeline level configuration, if specified overrides for this step


    model: {}


    # Same as pipeline level configuration, if specified overrides for this step


    settings:


    docker: {}


    resources: {}


    # Stack component specific settings


    step_operator.sagemaker:


    estimator_args:


    instance_type: m7g.medium


    Deep-dive


    enable_XXX parameters


    These are boolean flags for various configurations:


    enable_artifact_metadata: Whether to associate metadata with artifacts or not.


    enable_artifact_visualization: Whether to attach visualizations of artifacts.


    enable_cache: Utilize caching or not.


    enable_step_logs: Enable tracking step logs.


    enable_artifact_metadata: True


    enable_artifact_visualization: True


    enable_cache: True


    enable_step_logs: True


    build ID


    The UUID of the build to use for this pipeline. If specified, Docker image building
    is skipped for remote orchestrators, and the Docker image specified in this build
    is used.


    build: <INSERT-BUILD-ID-HERE>


    Configuring the model


    Specifies the ZenML Model to use for this pipeline.


    model:


    name: "ModelName"


    version: "production"


    description: An example model


    tags: ["classifier"]


    Pipeline and step parameters


    A dictionary of JSON-serializable parameters specified at the pipeline or step
    level. For example:


    parameters:


    gamma: 0.01


    steps:


    trainer:


    parameters:


    gamma: 0.001


    Corresponds to:


    from zenml import step, pipeline


    @step


    def trainer(gamma: float):


    # Use gamma as normal


    print(gamma)


    @pipeline


    def my_pipeline(gamma: float):


    # use gamma or pass it into the step


    print(0.01)


    trainer(gamma=gamma)'
- source_sentence: Can I use ZenML to register an S3 Artifact Store after setting
    up the IAM user and generating the access key?
  sentences:
  - ' to install and configure the AWS CLI on your hostyou don''t need to care about
    enabling your other stack components (orchestrators, step operators, and model
    deployers) to have access to the artifact store through IAM roles and policies


    you can combine the S3 artifact store with other stack components that are not
    running in AWS


    Note: When you create the IAM user for your AWS access key, please remember to
    grant the created IAM user permissions to read and write to your S3 bucket (i.e.
    at a minimum: s3:PutObject, s3:GetObject, s3:ListBucket, s3:DeleteObject)


    After having set up the IAM user and generated the access key, as described in
    the AWS documentation, you can register the S3 Artifact Store as follows:


    # Store the AWS access key in a ZenML secret


    zenml secret create s3_secret \


    --aws_access_key_id=''<YOUR_S3_ACCESS_KEY_ID>'' \


    --aws_secret_access_key=''<YOUR_S3_SECRET_KEY>''


    # Register the S3 artifact-store and reference the ZenML secret


    zenml artifact-store register s3_store -f s3 \


    --path=''s3://your-bucket'' \


    --authentication_secret=s3_secret


    # Register and set a stack with the new artifact store


    zenml stack register custom_stack -a s3_store ... --set


    Advanced Configuration


    The S3 Artifact Store accepts a range of advanced configuration options that can
    be used to further customize how ZenML connects to the S3 storage service that
    you are using. These are accessible via the client_kwargs, config_kwargs and s3_additional_kwargs
    configuration attributes and are passed transparently to the underlying S3Fs library:


    client_kwargs: arguments that will be transparently passed to the botocore client
    . You can use it to configure parameters like endpoint_url and region_name when
    connecting to an S3-compatible endpoint (e.g. Minio).


    config_kwargs: advanced parameters passed to botocore.client.Config.


    s3_additional_kwargs: advanced parameters that are used when calling S3 API, typically
    used for things like ServerSideEncryption and ACL.'
  - 'Evidently


    How to keep your data quality in check and guard against data and model drift
    with Evidently profiling


    The Evidently Data Validator flavor provided with the ZenML integration uses Evidently
    to perform data quality, data drift, model drift and model performance analyses,
    to generate reports and run checks. The reports and check results can be used
    to implement automated corrective actions in your pipelines or to render interactive
    representations for further visual interpretation, evaluation and documentation.


    When would you want to use it?


    Evidently is an open-source library that you can use to monitor and debug machine
    learning models by analyzing the data that they use through a powerful set of
    data profiling and visualization features, or to run a variety of data and model
    validation reports and tests, from data integrity tests that work with a single
    dataset to model evaluation tests to data drift analysis and model performance
    comparison tests. All this can be done with minimal configuration input from the
    user, or customized with specialized conditions that the validation tests should
    perform.


    Evidently currently works with tabular data in pandas.DataFrame or CSV file formats
    and can handle both regression and classification tasks.


    You should use the Evidently Data Validator when you need the following data and/or
    model validation features that are possible with Evidently:


    Data Quality reports and tests: provides detailed feature statistics and a feature
    behavior overview for a single dataset. It can also compare any two datasets.
    E.g. you can use it to compare train and test data, reference and current data,
    or two subgroups of one dataset.


    Data Drift reports and tests: helps detects and explore feature distribution changes
    in the input data by comparing two datasets with identical schema.'
  - ' us know!


    Configuration at pipeline or step levelWhen running your ZenML pipeline with the
    Sagemaker orchestrator, the configuration set when configuring the orchestrator
    as a ZenML component will be used by default. However, it is possible to provide
    additional configuration at the pipeline or step level. This allows you to run
    whole pipelines or individual steps with alternative configurations. For example,
    this allows you to run the training process with a heavier, GPU-enabled instance
    type, while running other steps with lighter instances.


    Additional configuration for the Sagemaker orchestrator can be passed via SagemakerOrchestratorSettings.
    Here, it is possible to configure processor_args, which is a dictionary of arguments
    for the Processor. For available arguments, see the Sagemaker documentation .
    Currently, it is not possible to provide custom configuration for the following
    attributes:


    image_uri


    instance_count


    sagemaker_session


    entrypoint


    base_job_name


    env


    For example, settings can be provided in the following way:


    sagemaker_orchestrator_settings = SagemakerOrchestratorSettings(


    processor_args={


    "instance_type": "ml.t3.medium",


    "volume_size_in_gb": 30


    They can then be applied to a step as follows:


    @step(settings={"orchestrator.sagemaker": sagemaker_orchestrator_settings})


    For example, if your ZenML component is configured to use ml.c5.xlarge with 400GB
    additional storage by default, all steps will use it except for the step above,
    which will use ml.t3.medium with 30GB additional storage.


    Check out this docs page for more information on how to specify settings in general.


    For more information and a full list of configurable attributes of the Sagemaker
    orchestrator, check out the SDK Docs .


    S3 data access in ZenML steps'
model-index:
- name: zenml/finetuned-all-MiniLM-L6-v2
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 384
      type: dim_384
    metrics:
    - type: cosine_accuracy@1
      value: 0.3313253012048193
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.6024096385542169
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.6927710843373494
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.7710843373493976
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.3313253012048193
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2008032128514056
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.13855421686746985
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.07710843373493974
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.3313253012048193
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.6024096385542169
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.6927710843373494
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.7710843373493976
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.551850042031417
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.4812392426850258
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.48905601510986996
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.3253012048192771
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.5903614457831325
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.6867469879518072
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.7710843373493976
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.3253012048192771
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.19678714859437746
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.13734939759036144
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.07710843373493974
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.3253012048192771
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.5903614457831325
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.6867469879518072
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.7710843373493976
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.5441628856415894
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.4714883342895392
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.4790082728748276
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.3313253012048193
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.5120481927710844
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.6144578313253012
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.6987951807228916
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.3313253012048193
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.1706827309236948
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.12289156626506023
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.06987951807228915
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.3313253012048193
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.5120481927710844
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.6144578313253012
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.6987951807228916
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.5108893388836802
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.45126936316695354
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.4605530012141939
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.2891566265060241
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.4759036144578313
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.5542168674698795
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.6265060240963856
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.2891566265060241
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.15863453815261044
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.1108433734939759
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.06265060240963854
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.2891566265060241
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.4759036144578313
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.5542168674698795
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.6265060240963856
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.45650145038804574
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.40227337923121054
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.4137670603435629
      name: Cosine Map@100
---

# zenml/finetuned-all-MiniLM-L6-v2

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision 8b3219a92973c328a8e22fadcfa821b5dc75636a -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("zenml/finetuned-all-MiniLM-L6-v2")
# Run inference
sentences = [
    'Can I use ZenML to register an S3 Artifact Store after setting up the IAM user and generating the access key?',
    " to install and configure the AWS CLI on your hostyou don't need to care about enabling your other stack components (orchestrators, step operators, and model deployers) to have access to the artifact store through IAM roles and policies\n\nyou can combine the S3 artifact store with other stack components that are not running in AWS\n\nNote: When you create the IAM user for your AWS access key, please remember to grant the created IAM user permissions to read and write to your S3 bucket (i.e. at a minimum: s3:PutObject, s3:GetObject, s3:ListBucket, s3:DeleteObject)\n\nAfter having set up the IAM user and generated the access key, as described in the AWS documentation, you can register the S3 Artifact Store as follows:\n\n# Store the AWS access key in a ZenML secret\n\nzenml secret create s3_secret \\\n\n--aws_access_key_id='<YOUR_S3_ACCESS_KEY_ID>' \\\n\n--aws_secret_access_key='<YOUR_S3_SECRET_KEY>'\n\n# Register the S3 artifact-store and reference the ZenML secret\n\nzenml artifact-store register s3_store -f s3 \\\n\n--path='s3://your-bucket' \\\n\n--authentication_secret=s3_secret\n\n# Register and set a stack with the new artifact store\n\nzenml stack register custom_stack -a s3_store ... --set\n\nAdvanced Configuration\n\nThe S3 Artifact Store accepts a range of advanced configuration options that can be used to further customize how ZenML connects to the S3 storage service that you are using. These are accessible via the client_kwargs, config_kwargs and s3_additional_kwargs configuration attributes and are passed transparently to the underlying S3Fs library:\n\nclient_kwargs: arguments that will be transparently passed to the botocore client . You can use it to configure parameters like endpoint_url and region_name when connecting to an S3-compatible endpoint (e.g. Minio).\n\nconfig_kwargs: advanced parameters passed to botocore.client.Config.\n\ns3_additional_kwargs: advanced parameters that are used when calling S3 API, typically used for things like ServerSideEncryption and ACL.",
    ' us know!\n\nConfiguration at pipeline or step levelWhen running your ZenML pipeline with the Sagemaker orchestrator, the configuration set when configuring the orchestrator as a ZenML component will be used by default. However, it is possible to provide additional configuration at the pipeline or step level. This allows you to run whole pipelines or individual steps with alternative configurations. For example, this allows you to run the training process with a heavier, GPU-enabled instance type, while running other steps with lighter instances.\n\nAdditional configuration for the Sagemaker orchestrator can be passed via SagemakerOrchestratorSettings. Here, it is possible to configure processor_args, which is a dictionary of arguments for the Processor. For available arguments, see the Sagemaker documentation . Currently, it is not possible to provide custom configuration for the following attributes:\n\nimage_uri\n\ninstance_count\n\nsagemaker_session\n\nentrypoint\n\nbase_job_name\n\nenv\n\nFor example, settings can be provided in the following way:\n\nsagemaker_orchestrator_settings = SagemakerOrchestratorSettings(\n\nprocessor_args={\n\n"instance_type": "ml.t3.medium",\n\n"volume_size_in_gb": 30\n\nThey can then be applied to a step as follows:\n\n@step(settings={"orchestrator.sagemaker": sagemaker_orchestrator_settings})\n\nFor example, if your ZenML component is configured to use ml.c5.xlarge with 400GB additional storage by default, all steps will use it except for the step above, which will use ml.t3.medium with 30GB additional storage.\n\nCheck out this docs page for more information on how to specify settings in general.\n\nFor more information and a full list of configurable attributes of the Sagemaker orchestrator, check out the SDK Docs .\n\nS3 data access in ZenML steps',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval
* Dataset: `dim_384`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.3313     |
| cosine_accuracy@3   | 0.6024     |
| cosine_accuracy@5   | 0.6928     |
| cosine_accuracy@10  | 0.7711     |
| cosine_precision@1  | 0.3313     |
| cosine_precision@3  | 0.2008     |
| cosine_precision@5  | 0.1386     |
| cosine_precision@10 | 0.0771     |
| cosine_recall@1     | 0.3313     |
| cosine_recall@3     | 0.6024     |
| cosine_recall@5     | 0.6928     |
| cosine_recall@10    | 0.7711     |
| cosine_ndcg@10      | 0.5519     |
| cosine_mrr@10       | 0.4812     |
| **cosine_map@100**  | **0.4891** |

#### Information Retrieval
* Dataset: `dim_256`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value     |
|:--------------------|:----------|
| cosine_accuracy@1   | 0.3253    |
| cosine_accuracy@3   | 0.5904    |
| cosine_accuracy@5   | 0.6867    |
| cosine_accuracy@10  | 0.7711    |
| cosine_precision@1  | 0.3253    |
| cosine_precision@3  | 0.1968    |
| cosine_precision@5  | 0.1373    |
| cosine_precision@10 | 0.0771    |
| cosine_recall@1     | 0.3253    |
| cosine_recall@3     | 0.5904    |
| cosine_recall@5     | 0.6867    |
| cosine_recall@10    | 0.7711    |
| cosine_ndcg@10      | 0.5442    |
| cosine_mrr@10       | 0.4715    |
| **cosine_map@100**  | **0.479** |

#### Information Retrieval
* Dataset: `dim_128`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.3313     |
| cosine_accuracy@3   | 0.512      |
| cosine_accuracy@5   | 0.6145     |
| cosine_accuracy@10  | 0.6988     |
| cosine_precision@1  | 0.3313     |
| cosine_precision@3  | 0.1707     |
| cosine_precision@5  | 0.1229     |
| cosine_precision@10 | 0.0699     |
| cosine_recall@1     | 0.3313     |
| cosine_recall@3     | 0.512      |
| cosine_recall@5     | 0.6145     |
| cosine_recall@10    | 0.6988     |
| cosine_ndcg@10      | 0.5109     |
| cosine_mrr@10       | 0.4513     |
| **cosine_map@100**  | **0.4606** |

#### Information Retrieval
* Dataset: `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.2892     |
| cosine_accuracy@3   | 0.4759     |
| cosine_accuracy@5   | 0.5542     |
| cosine_accuracy@10  | 0.6265     |
| cosine_precision@1  | 0.2892     |
| cosine_precision@3  | 0.1586     |
| cosine_precision@5  | 0.1108     |
| cosine_precision@10 | 0.0627     |
| cosine_recall@1     | 0.2892     |
| cosine_recall@3     | 0.4759     |
| cosine_recall@5     | 0.5542     |
| cosine_recall@10    | 0.6265     |
| cosine_ndcg@10      | 0.4565     |
| cosine_mrr@10       | 0.4023     |
| **cosine_map@100**  | **0.4138** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 1,490 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
  |         | positive                                                                          | anchor                                                                               |
  |:--------|:----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                               |
  | details | <ul><li>min: 9 tokens</li><li>mean: 21.16 tokens</li><li>max: 49 tokens</li></ul> | <ul><li>min: 21 tokens</li><li>mean: 239.82 tokens</li><li>max: 256 tokens</li></ul> |
* Samples:
  | positive                                                                                                                            | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
  |:------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Could you explain how to configure step-specific settings like the experiment_tracker and step_operator in ZenML docs?</code> | <code>docs.<br><br>failure_hook_source and success_hook_sourceThe source of the failure and success hooks can be specified.<br><br>Step-specific configuration<br><br>A lot of pipeline-level configuration can also be applied at a step level (as we have already seen with the enable_cache flag). However, there is some configuration that is step-specific, meaning it cannot be applied at a pipeline level, but only at a step level.<br><br>experiment_tracker: Name of the experiment_tracker to enable for this step. This experiment_tracker should be defined in the active stack with the same name.<br><br>step_operator: Name of the step_operator to enable for this step. This step_operator should be defined in the active stack with the same name.<br><br>outputs: This is configuration of the output artifacts of this step. This is further keyed by output name (by default, step outputs are named output). The most interesting configuration here is the materializer_source, which is the UDF path of the materializer in code to use for this output (e.g. materializers.some_data.materializer.materializer_class). Read more about this source path here.<br><br>PreviousHow to configure a pipeline with a YAML<br><br>NextRuntime settings for Docker, resources, and stack components<br><br>Last updated 19 days ago</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
  | <code>How do I configure ZenML to use the Azure Key Vault as a secrets store backend?</code>                                        | <code>COUNT_NAME>@<PROJECT_NAME>.iam.gserviceaccount.comUsing the Azure Key Vault as a secrets store backend<br><br>The Azure Secrets Store uses the ZenML Azure Service Connector under the hood to authenticate with the Azure Key Vault API. This means that you can use any of the authentication methods supported by the Azure Service Connector to authenticate with the Azure Key Vault API.<br><br>Example configuration for the Azure Key Vault Secrets Store:<br><br>zenml:<br><br># ...<br><br># Secrets store settings. This is used to store centralized secrets.<br><br>secretsStore:<br><br># Set to false to disable the secrets store.<br><br>enabled: true<br><br># The type of the secrets store<br><br>type: azure<br><br># Configuration for the Azure Key Vault secrets store<br><br>azure:<br><br># The name of the Azure Key Vault. This must be set to point to the Azure<br><br># Key Vault instance that you want to use.<br><br>key_vault_name:<br><br># The Azure Service Connector authentication method to use.<br><br>authMethod: service-principal<br><br># The Azure Service Connector configuration.<br><br>authConfig:<br><br># The Azure application service principal credentials to use to<br><br># authenticate with the Azure Key Vault API.<br><br>client_id: <your Azure client ID><br><br>client_secret: <your Azure client secret><br><br>tenant_id: <your Azure tenant ID><br><br>Using the HashiCorp Vault as a secrets store backend<br><br>To use the HashiCorp Vault service as a Secrets Store back-end, it must be configured in the Helm values:<br><br>zenml:<br><br># ...<br><br># Secrets store settings. This is used to store centralized secrets.<br><br>secretsStore:<br><br># Set to false to disable the secrets store.<br><br>enabled: true<br><br># The type of the secrets store<br><br>type: hashicorp<br><br># Configuration for the HashiCorp Vault secrets store<br><br>hashicorp:<br><br># The url of the HashiCorp Vault server to use<br><br>vault_addr: https://vault.example.com<br><br># The token used to authenticate with the Vault server<br><br>vault_token: <your Vault token><br><br># The Vault Enterprise namespace. Not required for Vault OSS.<br><br>vault_namespace: <your Vault namespace><br><br>Using a custom secrets store backend implementation</code> |
  | <code>How do I register a service connector with AWS using ZenML?</code>                                                            | <code>r the local cloud provider CLI (AWS in this case):AWS_PROFILE=connectors zenml service-connector register aws-sts-token --type aws --auto-configure --auth-method sts-token<br><br>Example Command Output<br><br>⠸ Registering service connector 'aws-sts-token'...<br><br>Successfully registered service connector `aws-sts-token` with access to the following resources:<br><br>┏━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓<br><br>┃     RESOURCE TYPE     │ RESOURCE NAMES                               ┃<br><br>┠───────────────────────┼──────────────────────────────────────────────┨<br><br>┃    🔶 aws-generic     │ us-east-1                                    ┃<br><br>┠───────────────────────┼──────────────────────────────────────────────┨<br><br>┃     📦 s3-bucket      │ s3://zenfiles                                ┃<br><br>┃                       │ s3://zenml-demos                             ┃<br><br>┃                       │ s3://zenml-generative-chat                   ┃<br><br>┃                       │ s3://zenml-public-datasets                   ┃<br><br>┠───────────────────────┼──────────────────────────────────────────────┨<br><br>┃ 🌀 kubernetes-cluster │ zenhacks-cluster                             ┃<br><br>┠───────────────────────┼──────────────────────────────────────────────┨<br><br>┃  🐳 docker-registry   │ 715803424590.dkr.ecr.us-east-1.amazonaws.com ┃<br><br>┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛<br><br>The Service Connector is now configured with a short-lived token that will expire after some time. You can verify this by inspecting the Service Connector:<br><br>zenml service-connector describe aws-sts-token<br><br>Example Command Output<br><br>Service connector 'aws-sts-token' of type 'aws' with id '63e14350-6719-4255-b3f5-0539c8f7c303' is owned by user 'default' and is 'private'.<br><br>'aws-sts-token' aws Service Connector Details<br><br>┏━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓<br><br>┃ PROPERTY         │ VALUE                                                                   ┃</code>                                                                                                                       |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          384,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: True
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch      | Step  | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_384_cosine_map@100 | dim_64_cosine_map@100 |
|:----------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 0.6667     | 1     | 0.4231                 | 0.4458                 | 0.4525                 | 0.3907                |
| 2.0        | 3     | 0.4526                 | 0.4745                 | 0.4875                 | 0.4114                |
| **2.6667** | **4** | **0.4606**             | **0.479**              | **0.4891**             | **0.4138**            |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.10.14
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.3.1+cu121
- Accelerate: 0.31.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->