diff --git "a/README.md" "b/README.md" --- "a/README.md" +++ "b/README.md" @@ -31,1060 +31,1145 @@ tags: - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: -- source_sentence: What are some examples of long-lived credentials used in ZenML - for authentication? +- source_sentence: Can you explain how to configure the credentials for authentication + to a remote MLflow tracking server in ZenML? sentences: - - ' gs://zenml-core_cloudbuild ┃┃ │ gs://zenml-datasets ┃ + - 'w_bucket=gs://my_bucket --provider=You can pass other configurations + specific to the stack components as key-value arguments. If you don''t provide + a name, a random one is generated for you. For more information about how to work + use the CLI for this, please refer to the dedicated documentation section. - ┃ │ gs://zenml-internal-artifact-store ┃ + Authentication Methods - ┃ │ gs://zenml-kubeflow-artifact-store ┃ + You need to configure the following credentials for authentication to a remote + MLflow tracking server: - ┃ │ gs://zenml-project-time-series-bucket ┃ + tracking_uri: The URL pointing to the MLflow tracking server. If using an MLflow + Tracking Server managed by Databricks, then the value of this attribute should + be "databricks". - ┠───────────────────────┼─────────────────────────────────────────────────┨ + tracking_username: Username for authenticating with the MLflow tracking server. - ┃ 🌀 kubernetes-cluster │ zenml-test-cluster ┃ + tracking_password: Password for authenticating with the MLflow tracking server. - ┠───────────────────────┼─────────────────────────────────────────────────┨ + tracking_token (in place of tracking_username and tracking_password): Token for + authenticating with the MLflow tracking server. - ┃ 🐳 docker-registry │ gcr.io/zenml-core ┃ + tracking_insecure_tls (optional): Set to skip verifying the MLflow tracking server + SSL certificate. - ┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ + databricks_host: The host of the Databricks workspace with the MLflow-managed + server to connect to. This is only required if the tracking_uri value is set to + "databricks". More information: Access the MLflow tracking server from outside + Databricks - Long-lived credentials (API keys, account keys) + Either tracking_token or tracking_username and tracking_password must be specified. - This is the magic formula of authentication methods. When paired with another - ability, such as automatically generating short-lived API tokens, or impersonating - accounts or assuming roles, this is the ideal authentication mechanism to use, - particularly when using ZenML in production and when sharing results with other - members of your ZenML team. + This option configures the credentials for the MLflow tracking service directly + as stack component attributes. - As a general best practice, but implemented particularly well for cloud platforms, - account passwords are never directly used as a credential when authenticating - to the cloud platform APIs. There is always a process in place that exchanges - the account/password credential for another type of long-lived credential: + This is not recommended for production settings as the credentials won''t be stored + securely and will be clearly visible in the stack configuration. - AWS uses the aws configure CLI command + # Register the MLflow experiment tracker - GCP offers the gcloud auth application-default login CLI commands + zenml experiment-tracker register mlflow_experiment_tracker --flavor=mlflow \ - Azure provides the az login CLI command + --tracking_uri= --tracking_token= - None of your original login information is stored on your local machine or used - to access workloads. Instead, an API key, account key or some other form of intermediate - credential is generated and stored on the local host and used to authenticate - to remote cloud service APIs.' - - '─────────────────────────────────────────────────┨┃ VERSION │ - 1 ┃ + # You can also register it like this: - ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨ + # zenml experiment-tracker register mlflow_experiment_tracker --flavor=mlflow + \ - ┃ VERSION_DESCRIPTION │ Run #1 of the mlflow_training_pipeline. ┃ + # --tracking_uri= --tracking_username= --tracking_password= - ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨ + # Register and set a stack with the new experiment tracker' + - 'token_hex + token_hex(32)or:Copyopenssl rand -hex 32Important: If you configure encryption + for your SQL database secrets store, you should keep the ZENML_SECRETS_STORE_ENCRYPTION_KEY + value somewhere safe and secure, as it will always be required by the ZenML server + to decrypt the secrets in the database. If you lose the encryption key, you will + not be able to decrypt the secrets in the database and will have to reset them. - ┃ CREATED_AT │ 2023-03-01 09:09:06.899000 ┃ + These configuration options are only relevant if you''re using the AWS Secrets + Manager as the secrets store backend. - ┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┨ + ZENML_SECRETS_STORE_TYPE: Set this to aws in order to set this type of secret + store. - ┃ UPDATED_AT │ 2023-03-01 09:09:06.899000 ┃' - - 'un.py + The AWS Secrets Store uses the ZenML AWS Service Connector under the hood to authenticate + with the AWS Secrets Manager API. This means that you can use any of the authentication + methods supported by the AWS Service Connector to authenticate with the AWS Secrets + Manager API. - Read more in the production guide. + "Version": "2012-10-17", - CleanupMake sure you no longer need the resources before deleting them. The instructions - and commands that follow are DESTRUCTIVE. + "Statement": [ - Delete any AWS resources you no longer use to avoid additional charges. You''ll - want to do the following: + "Sid": "ZenMLSecretsStore", - # delete the S3 bucket + "Effect": "Allow", - aws s3 rm s3://your-bucket-name --recursive + "Action": [ - aws s3api delete-bucket --bucket your-bucket-name + "secretsmanager:CreateSecret", - # delete the SageMaker domain + "secretsmanager:GetSecretValue", - aws sagemaker delete-domain --domain-id + "secretsmanager:DescribeSecret", - # delete the ECR repository + "secretsmanager:PutSecretValue", - aws ecr delete-repository --repository-name zenml-repository --force + "secretsmanager:TagResource", - # detach policies from the IAM role + "secretsmanager:DeleteSecret" - aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess + ], - aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess + "Resource": "arn:aws:secretsmanager:::secret:zenml/*" - aws iam detach-role-policy --role-name zenml-role --policy-arn arn:aws:iam::aws:policy/AmazonSageMakerFullAccess + The following configuration options are supported: - # delete the IAM role + ZENML_SECRETS_STORE_AUTH_METHOD: The AWS Service Connector authentication method + to use (e.g. secret-key or iam-role). - aws iam delete-role --role-name zenml-role + ZENML_SECRETS_STORE_AUTH_CONFIG: The AWS Service Connector configuration, in JSON + format (e.g. {"aws_access_key_id":"","aws_secret_access_key":"","region":""}). - Make sure to run these commands in the same AWS region where you created the resources. + Note: The remaining configuration options are deprecated and may be removed in + a future release. Instead, you should set the ZENML_SECRETS_STORE_AUTH_METHOD + and ZENML_SECRETS_STORE_AUTH_CONFIG variables to use the AWS Service Connector + authentication method.' + - 'tive Directory credentials or generic OIDC tokens.This authentication method + only requires a GCP workload identity external account JSON file that only contains + the configuration for the external account without any sensitive credentials. + It allows implementing a two layer authentication scheme that keeps the set of + permissions associated with implicit credentials down to the bare minimum and + grants permissions to the privilege-bearing GCP service account instead. - By running these cleanup commands, you will delete the S3 bucket, SageMaker domain, - ECR repository, and IAM role, along with their associated policies. This will - help you avoid any unnecessary charges for resources you no longer need. + This authentication method can be used to authenticate to GCP services using credentials + from other cloud providers or identity providers. When used with workloads running + on AWS or Azure, it involves automatically picking up credentials from the AWS + IAM or Azure AD identity associated with the workload and using them to authenticate + to GCP services. This means that the result depends on the environment where the + ZenML server is deployed and is thus not fully reproducible. - Remember to be cautious when deleting resources and ensure that you no longer - require them before running the deletion commands. + When used with AWS or Azure implicit in-cloud authentication, this method may + constitute a security risk, because it can give users access to the identity (e.g. + AWS IAM role or Azure AD principal) implicitly associated with the environment + where the ZenML server is running. For this reason, all implicit authentication + methods are disabled by default and need to be explicitly enabled by setting the + ZENML_ENABLE_IMPLICIT_AUTH_METHODS environment variable or the helm chart enableImplicitAuthMethods + configuration option to true in the ZenML deployment. - Conclusion + + By default, the GCP connector generates temporary OAuth 2.0 tokens from the external + account credentials and distributes them to clients. The tokens have a limited + lifetime of 1 hour. This behavior can be disabled by setting the generate_temporary_tokens + configuration option to False, in which case, the connector will distribute the + external account credentials JSON to clients instead (not recommended).' +- source_sentence: What is an example of a ZenML server YAML configuration file? + sentences: + - 'sing a type annotation. - In this guide, we walked through the process of setting up an AWS stack with ZenML - to run your machine learning pipelines in a scalable and production-ready environment. - The key steps included: + Tuple vs multiple outputsIt is impossible for ZenML to detect whether you want + your step to have a single output artifact of type Tuple or multiple output artifacts + just by looking at the type annotation. - Setting up credentials and the local environment by creating an IAM role with - the necessary permissions. + We use the following convention to differentiate between the two: When the return + statement is followed by a tuple literal (e.g. return 1, 2 or return (value_1, + value_2)) we treat it as a step with multiple outputs. All other cases are treated + as a step with a single output of type Tuple. - Creating a ZenML service connector to authenticate with AWS services using the - IAM role. + from zenml import step - Configuring stack components, including an S3 artifact store, a SageMaker Pipelines - orchestrator, and an ECR container registry.' -- source_sentence: Can you provide more information on how to fetch the last successful - run of a pipeline in ZenML? - sentences: - - 'Connecting remote storage + from typing_extensions import Annotated - Transitioning to remote artifact storage. + from typing import Tuple - In the previous chapters, we''ve been working with artifacts stored locally on - our machines. This setup is fine for individual experiments, but as we move towards - a collaborative and production-ready environment, we need a solution that is more - robust, shareable, and scalable. Enter remote storage! + # Single output artifact - Remote storage allows us to store our artifacts in the cloud, which means they''re - accessible from anywhere and by anyone with the right permissions. This is essential - for team collaboration and for managing the larger datasets and models that come - with production workloads. + @step - When using a stack with remote storage, nothing changes except the fact that the - artifacts get materialized in a central and remote storage location. This diagram - explains the flow: + def my_step() -> Tuple[int, int]: - Provisioning and registering a remote artifact store + output_value = (0, 1) - Out of the box, ZenML ships with many different supported artifact store flavors. - For convenience, here are some brief instructions on how to quickly get up and - running on the major cloud providers: + return output_value - You will need to install and set up the AWS CLI on your machine as a prerequisite, - as covered in the AWS CLI documentation, before you register the S3 Artifact Store. + # Single output artifact with variable length - The Amazon Web Services S3 Artifact Store flavor is provided by the S3 ZenML integration, - you need to install it on your local machine to be able to register an S3 Artifact - Store and add it to your stack: + @step - zenml integration install s3 -y + def my_step(condition) -> Tuple[int, ...]: - Having trouble with this command? You can use poetry or pip to install the requirements - of any ZenML integration directly. In order to obtain the exact requirements of - the AWS S3 integration you can use zenml integration requirements s3. + if condition: - The only configuration parameter mandatory for registering an S3 Artifact Store - is the root path URI, which needs to point to an S3 bucket and take the form s3://bucket-name. - In order to create a S3 bucket, refer to the AWS documentation.' - - 'Load a Model in code + output_value = (0, 1) - There are a few different ways to load a ZenML Model in code: + else: - Load the active model in a pipeline + output_value = (0, 1, 2) - You can also use the active model to get the model metadata, or the associated - artifacts directly as described in the starter guide: + return output_value - from zenml import step, pipeline, get_step_context, pipeline, Model + # Single output artifact using the `Annotated` annotation - @pipeline(model=Model(name="my_model")) + @step - def my_pipeline(): + def my_step() -> Annotated[Tuple[int, ...], "my_output"]: - ... + return 0, 1 + + + # Multiple output artifacts @step - def my_step(): + def my_step() -> Tuple[int, int]: + + + return 0, 1 - # Get model from active step context + # Not allowed: Variable length tuple annotation when using - mv = get_step_context().model + # multiple output artifacts - # Get metadata + @step + + def my_step() -> Tuple[int, ...]: - print(mv.run_metadata["metadata_key"].value) + return 0, 1 - # Directly fetch an artifact that is attached to the model + Step output names - output = mv.get_artifact("my_dataset", "my_version") + By default, ZenML uses the output name output for single output steps and output_0, + output_1, ... for steps with multiple outputs. These output names are used to + display your outputs in the dashboard and fetch them after your pipeline is finished. - output.run_metadata["accuracy"].value + If you want to use custom output names for your steps, use the Annotated type + annotation: - Load any model via the Client + from typing_extensions import Annotated # or `from typing import Annotated on + Python 3.9+ - Alternatively, you can use the Client: + + from typing import Tuple from zenml import step - from zenml.client import Client + @step + + + def square_root(number: int) -> Annotated[float, "custom_output_name"]: - from zenml.enums import ModelStages + return number ** 0.5 @step - def model_evaluator_step() + def divide(a: int, b: int) -> Tuple[ - ... + Annotated[int, "quotient"], - # Get staging model version + Annotated[int, "remainder"] - try: + ]: - staging_zenml_model = Client().get_model_version( + return a // b, a % b' + - 'HyperAI Orchestrator - model_name_or_id="", + Orchestrating your pipelines to run on HyperAI.ai instances. - model_version_name_or_number_or_id=ModelStages.STAGING, + HyperAI is a cutting-edge cloud compute platform designed to make AI accessible + for everyone. The HyperAI orchestrator is an orchestrator flavor that allows you + to easily deploy your pipelines on HyperAI instances. - except KeyError: + This component is only meant to be used within the context of a remote ZenML deployment + scenario. Usage with a local ZenML deployment may lead to unexpected behavior! - staging_zenml_model = None + When to use it - ... + You should use the HyperAI orchestrator if: - PreviousControlling Model versions + you''re looking for a managed solution for running your pipelines. - NextPromote a Model + you''re a HyperAI customer. - Last updated 19 days ago' - - ' more information. + Prerequisites - Get the last run of a pipelineTo access the most recent run of a pipeline, you - can either use the last_run property or access it through the runs list: + You will need to do the following to start using the HyperAI orchestrator: - last_run = pipeline_model.last_run # OR: pipeline_model.runs[0] + Have a running HyperAI instance. It must be accessible from the internet (or at + least from the IP addresses of your ZenML users) and allow SSH key based access + (passwords are not supported). - If your most recent runs have failed, and you want to find the last run that has - succeeded, you can use the last_successful_run property instead. + Ensure that a recent version of Docker is installed. This version must include + Docker Compose, meaning that the command docker compose works. - Get the latest run from a pipeline + Ensure that the appropriate NVIDIA Driver is installed on the HyperAI instance + (if not already installed by the HyperAI team). - Calling a pipeline executes it and then returns the response of the freshly executed - run. + Ensure that the NVIDIA Container Toolkit is installed and configured on the HyperAI + instance. - run = training_pipeline() + Note that it is possible to omit installing the NVIDIA Driver and NVIDIA Container + Toolkit. However, you will then be unable to use the GPU from within your ZenML + pipeline. Additionally, you will then need to disable GPU access within the container + when configuring the Orchestrator component, or the pipeline will not start correctly. - The run that you get back is the model stored in the ZenML database at the point - of the method call. This means the pipeline run is still initializing and no steps - have been run. To get the latest state can get a refreshed version from the client: + How it works' + - 'fied, or a string, in which case it must be a path# to a CA certificate bundle + to use or the CA bundle value itself - from zenml.client import Client + verify_ssl: - Client().get_pipeline_run(run.id) to get a refreshed version + Here is an example of a ZenML server YAML configuration file: - Get a run via the client + url: https://ac8ef63af203226194a7725ee71d85a-7635928635.us-east-1.elb.amazonaws.com/zenml - If you already know the exact run that you want to fetch (e.g., from looking at - the dashboard), you can use the Client.get_pipeline_run() method to fetch the - run directly without having to query the pipeline first: + verify_ssl: | - from zenml.client import Client + -----BEGIN CERTIFICATE----- - pipeline_run = Client().get_pipeline_run("first_pipeline-2023_06_20-16_20_13_274466") + ... - Similar to pipelines, you can query runs by either ID, name, or name prefix, and - you can also discover runs through the Client or CLI via the Client.list_pipeline_runs() - or zenml pipeline runs list commands. + -----END CERTIFICATE----- - Run information + To disconnect from the current ZenML server and revert to using the local default + database, use the following command: - Each run has a collection of useful information which can help you reproduce your - runs. In the following, you can find a list of some of the most useful pipeline - run information, but there is much more available. See the PipelineRunResponse - definition for a comprehensive list. + zenml disconnect - Status + How does it work? - The status of a pipeline run. There are five possible states: initialized, failed, - completed, running, and cached. + Here''s an architecture diagram that shows how the workflow looks like when you + do zenml deploy. - status = run.status + The deploy CLI makes use of a "recipe" inside the zenml-io/zenml repository to + deploy the server on the right cloud. Any configuration that you pass with the + CLI, is sent to the recipe as input variables. - Configuration' -- source_sentence: How does the code in the given documentation visualize the embeddings - using t-SNE and UMAP? - sentences: - - ' + PreviousDeploying ZenML - embeddings_2d[mask, 0], + NextDeploy with Docker - embeddings_2d[mask, 1],c=[section_color_dict[section]], + Last updated 15 days ago' +- source_sentence: When should I update my service account name to ensure security? + sentences: + - 'y update. - label=section, + Important noticeEvery API key issued is a potential gateway to access your data, + secrets and infrastructure. It''s important to regularly rotate API keys and deactivate + or delete service accounts and API keys that are no longer needed. - plt.title("t-SNE Visualization") + PreviousConnect in with your User (interactive) - plt.legend() + NextInteract with secrets - plt.show() + Last updated 15 days ago' + - 'Connect in with your User (interactive) - # Dimensionality reduction using UMAP + You can authenticate your clients with the ZenML Server using the ZenML CLI and + the web based login. This can be executed with the command: - def umap_visualization(embeddings, parent_sections): + zenml connect --url https://... - umap_2d = umap.UMAP(n_components=2, random_state=42) + This command will start a series of steps to validate the device from where you + are connecting that will happen in your browser. You can choose whether to mark + your respective device as trusted or not. If you choose not to click Trust this + device, a 24-hour token will be issued for authentication services. Choosing to + trust the device will issue a 30-day token instead. - embeddings_2d = umap_2d.fit_transform(embeddings) + To see all devices you''ve permitted, use the following command: - plt.figure(figsize=(8, 8)) + zenml authorized-device list - for section in unique_parent_sections: + Additionally, the following command allows you to more precisely inspect one of + these devices: - if section in section_color_dict: + zenml authorized-device describe - mask = [section == ps for ps in parent_sections] + For increased security, you can invalidate a token using the zenml device lock + command followed by the device ID. This helps provide an extra layer of security + and control over your devices. - plt.scatter( + zenml authorized-device lock - embeddings_2d[mask, 0], + To keep things simple, we can summarize the steps: - embeddings_2d[mask, 1], + Use the zenml connect --url command to start a device flow and connect to a zenml + server. - c=[section_color_dict[section]], + Choose whether to trust the device when prompted. - label=section, + Check permitted devices with zenml devices list. - plt.title("UMAP Visualization") + Invalidate a token with zenml device lock .... - plt.legend() + Important notice - plt.show() + Using the ZenML CLI is a secure and comfortable way to interact with your ZenML + tenants. It''s important to always ensure that only trusted devices are used to + maintain security and privacy. - In this stage, we have utilized the ''parent directory'', which we had previously - stored in the vector store as an additional attribute, as a means to color the - values. This approach allows us to gain some insight into the semantic space inherent - in our data. It demonstrates that you can visualize the embeddings and observe - how similar chunks are grouped together based on their semantic meaning and context. + Don''t forget to manage your device trust levels regularly for optimal security. + Should you feel a device trust needs to be revoked, lock the device immediately. + Every token issued is a potential gateway to access your data, secrets and infrastructure. - So this step iterates through all the chunks and generates embeddings representing - each piece of text. These embeddings are then stored as an artifact in the ZenML - artifact store as a NumPy array. We separate this generation from the point where - we upload those embeddings to the vector database to keep the pipeline modular - and flexible; in the future we might want to use a different vector database so - we can just swap out the upload step without having to re-generate the embeddings. + PreviousConnect to a server - In the next section, we''ll explore how to store these embeddings in a vector - database to enable fast and efficient retrieval of relevant chunks at inference - time. + NextConnect with a Service Account - Code Example + Last updated 19 days ago' + - '━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━┷━━━━━━━━┛A lot more is hidden behind + a Service Connector Type than a name and a simple list of resource types. Before + using a Service Connector Type to configure a Service Connector, you probably + need to understand what it is, what it can offer and what are the supported authentication + methods and their requirements. All this can be accessed directly through the + CLI. Some examples are included here. - To explore the full code, visit the Complete Guide repository. The embeddings - generation step can be found here. + Showing information about the gcp Service Connector Type: - PreviousData ingestion and preprocessing + zenml service-connector describe-type gcp - NextStoring embeddings in a vector database + Example Command Output - Last updated 2 months ago' - - 'Amazon SageMaker + ╔══════════════════════════════════════════════════════════════════════════════╗ - Executing individual steps in SageMaker. + ║ 🔵 GCP Service Connector (connector type: gcp) ║ - SageMaker offers specialized compute instances to run your training jobs and has - a comprehensive UI to track and manage your models and logs. ZenML''s SageMaker - step operator allows you to submit individual steps to be run on Sagemaker compute - instances. + ╚══════════════════════════════════════════════════════════════════════════════╝ - When to use it + Authentication methods: - You should use the SageMaker step operator if: + 🔒 implicit - one or more steps of your pipeline require computing resources (CPU, GPU, memory) - that are not provided by your orchestrator. + 🔒 user-account - you have access to SageMaker. If you''re using a different cloud provider, take - a look at the Vertex or AzureML step operators. + 🔒 service-account - How to deploy it + 🔒 oauth2-token - Create a role in the IAM console that you want the jobs running in SageMaker to - assume. This role should at least have the AmazonS3FullAccess and AmazonSageMakerFullAccess - policies applied. Check here for a guide on how to set up this role. + 🔒 impersonation - Infrastructure Deployment + Resource types: - A Sagemaker step operator can be deployed directly from the ZenML CLI: + 🔵 gcp-generic - zenml orchestrator deploy sagemaker_step_operator --flavor=sagemaker --provider=aws - ... + 📦 gcs-bucket - You can pass other configurations specific to the stack components as key-value - arguments. If you don''t provide a name, a random one is generated for you. For - more information about how to work use the CLI for this, please refer to the dedicated - documentation section. + 🌀 kubernetes-cluster - How to use it + 🐳 docker-registry - To use the SageMaker step operator, we need: + Supports auto-configuration: True - The ZenML aws integration installed. If you haven''t done so, runCopyzenml integration - install aws + Available locally: True - Docker installed and running. + Available remotely: True + + The ZenML GCP Service Connector facilitates the authentication and access to - An IAM role with the correct permissions. See the deployment section for detailed - instructions. + managed GCP services and resources. These encompass a range of resources, - An AWS container registry as part of our stack. Take a look here for a guide on - how to set that up.' - - 'Deploy with Docker + including GCS buckets, GCR container repositories and GKE clusters. The - Deploying ZenML in a Docker container. + connector provides support for various authentication methods, including GCP - The ZenML server container image is available at zenmldocker/zenml-server and - can be used to deploy ZenML with a container management or orchestration tool - like Docker and docker-compose, or a serverless platform like Cloud Run, Container - Apps, and more! This guide walks you through the various configuration options - that the ZenML server container expects as well as a few deployment use cases. + user accounts, service accounts, short-lived OAuth 2.0 tokens and implicit - Try it out locally first + authentication. - If you''re just looking for a quick way to deploy the ZenML server using a container, - without going through the hassle of interacting with a container management tool - like Docker and manually configuring your container, you can use the ZenML CLI - to do so. You only need to have Docker installed and running on your machine: + To ensure heightened security measures, this connector always issues short-lived - zenml up --docker + OAuth 2.0 tokens to clients instead of long-lived credentials. Furthermore, it - This command deploys a ZenML server locally in a Docker container, then connects - your client to it. Similar to running plain zenml up, the server and the local - ZenML client share the same SQLite database. + includes automatic configuration and detection of credentials locally - The rest of this guide is addressed to advanced users who are looking to manually - deploy and manage a containerized ZenML server. + configured through the GCP CLI. - ZenML server configuration options + This connector serves as a general means of accessing any GCP service by issuing - If you''re planning on deploying a custom containerized ZenML server yourself, - you probably need to configure some settings for it like the database it should - use, the default user details, and more. The ZenML server container image uses - sensible defaults, so you can simply start a container without worrying too much - about the configuration. However, if you''re looking to connect the ZenML server - to an external MySQL database or secrets management service, to persist the internal - SQLite database, or simply want to control other settings like the default account, - you can do so by customizing the container''s environment variables. + OAuth 2.0 credential objects to clients. Additionally, the connector can handle - The following environment variables can be passed to the container:' -- source_sentence: What is the purpose of the `enable_step_logs` parameter in ZenML? + + specialized authentication for GCS, Docker and Kubernetes Python clients. It' +- source_sentence: Where can I find the instructions to clone the ZenML quickstart + repository and set up the stack? sentences: - - 'Collect information from your SQL database serviceUsing an external MySQL-compatible - database service is optional, but is recommended for production deployments. If - omitted, ZenML will default to using an embedded SQLite database, which has the - following limitations: + - 'into play when the component is ultimately in use.The design behind this interaction + lets us separate the configuration of the flavor from its implementation. This + way we can register flavors and components even when the major dependencies behind + their implementation are not installed in our local setting (assuming the CustomArtifactStoreFlavor + and the CustomArtifactStoreConfig are implemented in a different module/path than + the actual CustomArtifactStore). + + Enabling Artifact Visualizations with Custom Artifact Stores - the SQLite database is not persisted, meaning that it will be lost if the ZenML - server pod is restarted or deleted + ZenML automatically saves visualizations for many common data types and allows + you to view these visualizations in the ZenML dashboard. Under the hood, this + works by saving the visualizations together with the artifacts in the artifact + store. - the SQLite database does not scale horizontally, meaning that you will not be - able to use more than one replica at a time for the ZenML server pod + In order to load and display these visualizations, ZenML needs to be able to load + and access the corresponding artifact store. This means that your custom artifact + store needs to be configured in a way that allows authenticating to the back-end + without relying on the local environment, e.g., by embedding the authentication + credentials in the stack component configuration or by referencing a secret. - If you decide to use an external MySQL-compatible database service, you will need - to collect and prepare the following information for the Helm chart configuration: + Furthermore, for deployed ZenML instances, you need to install the package dependencies + of your artifact store implementation in the environment where you have deployed + ZenML. See the Documentation on deploying ZenML with custom Docker images for + more information on how to do that. - the hostname and port where the SQL database is reachable from the Kubernetes - cluster + PreviousAzure Blob Storage - the username and password that will be used to connect to the database. It is - recommended that you create a dedicated database user for the ZenML server and - that you restrict its privileges to only access the database that will be used - by ZenML. Enforcing secure SSL connections for the user/database is also recommended. - See the MySQL documentation for more information on how to set up users and privileges. + NextContainer Registries - the name of the database that will be used by ZenML. The database does not have - to exist prior to the deployment ( ZenML will create it on the first start). However, - you need to create the database if you follow the best practice of restricting - database user privileges to only access it. + Last updated 19 days ago' + - 't_repository: str - if you plan on using SSL to secure the client database connection, you may also - need to prepare additional SSL certificates and keys:the TLS CA certificate that - was used to sign the server TLS certificate, if you''re using a self-signed certificate - or signed by a custom certificate authority that is not already trusted by default - by most operating systems.the TLS client certificate and key. This is only needed - if you decide to use client certificates for your DB connection (some managed - DB services support this, CloudSQL is an example).' - - ' │ SHARED │ OWNER │ EXPIRES IN │ LABELS ┃┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨ + user: Optional[str] - ┃ │ gcp-multi │ 9d953320-3560-4a78-817c-926a3898064d │ 🔵 gcp - │ 🔵 gcp-generic │ │ ➖ │ default │ │ ┃ + resources:cpu_count: Optional[PositiveFloat] - ┃ │ │ │ │ - 📦 gcs-bucket │ │ │ │ │ ┃ + gpu_count: Optional[NonNegativeInt] - ┃ │ │ │ │ - 🌀 kubernetes-cluster │ │ │ │ │ ┃ + memory: Optional[ConstrainedStrValue] - ┃ │ │ │ │ - 🐳 docker-registry │ │ │ │ │ ┃ + step_operator: Optional[str] - ┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨ + success_hook_source: - ┃ │ gcs-multi │ ff9c0723-7451-46b7-93ef-fcf3efde30fa │ 🔵 gcp - │ 📦 gcs-bucket │ │ ➖ │ default │ │ ┃ + attribute: Optional[str] - ┠────────┼────────────────────────┼──────────────────────────────────────┼────────┼───────────────────────┼─────────────────────────┼────────┼─────────┼────────────┼────────┨ + module: str - ┃ │ gcs-langchain-slackbot │ cf3953e9-414c-4875-ba00-24c62a0dc0c5 │ 🔵 gcp - │ 📦 gcs-bucket │ gs://langchain-slackbot │ ➖ │ default │ │ ┃ + type: SourceType - ┗━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━┛ + train_model: - Local and remote availability' - - 'onfiguration, if specified overrides for this stepenable_artifact_metadata: True + enable_artifact_metadata: Optional[bool] - enable_artifact_visualization: True + enable_artifact_visualization: Optional[bool] - enable_cache: False + enable_cache: Optional[bool] - enable_step_logs: True + enable_step_logs: Optional[bool] - # Same as pipeline level configuration, if specified overrides for this step + experiment_tracker: Optional[str] - extra: {} + extra: Mapping[str, Any] - # Same as pipeline level configuration, if specified overrides for this step + failure_hook_source: - model: {} + attribute: Optional[str] - # Same as pipeline level configuration, if specified overrides for this step + module: str - settings: + type: SourceType - docker: {} + model: - resources: {} + audience: Optional[str] - # Stack component specific settings + description: Optional[str] - step_operator.sagemaker: + ethics: Optional[str] - estimator_args: + license: Optional[str] - instance_type: m7g.medium + limitations: Optional[str] - Deep-dive + name: str - enable_XXX parameters + save_models_to_registry: bool - These are boolean flags for various configurations: + suppress_class_validation_warnings: bool - enable_artifact_metadata: Whether to associate metadata with artifacts or not. + tags: Optional[List[str]] - enable_artifact_visualization: Whether to attach visualizations of artifacts. + trade_offs: Optional[str] - enable_cache: Utilize caching or not. + use_cases: Optional[str] - enable_step_logs: Enable tracking step logs. + version: Union[ModelStages, int, str, NoneType] - enable_artifact_metadata: True + was_created_in_this_run: bool - enable_artifact_visualization: True + name: Optional[str] - enable_cache: True + outputs: {} - enable_step_logs: True + parameters: {} - build ID + settings: - The UUID of the build to use for this pipeline. If specified, Docker image building - is skipped for remote orchestrators, and the Docker image specified in this build - is used. + docker: - build: + apt_packages: List[str] - Configuring the model + build_context_root: Optional[str] - Specifies the ZenML Model to use for this pipeline. + build_options: Mapping[str, Any] - model: + copy_files: bool - name: "ModelName" + copy_global_config: bool - version: "production" + dockerfile: Optional[str] - description: An example model + dockerignore: Optional[str] - tags: ["classifier"] + environment: Mapping[str, Any] - Pipeline and step parameters + install_stack_requirements: bool - A dictionary of JSON-serializable parameters specified at the pipeline or step - level. For example: + parent_image: Optional[str] - parameters: + python_package_installer: PythonPackageInstaller - gamma: 0.01 + replicate_local_python_environment: Union[List[str], PythonEnvironmentExportMethod, - steps: + NoneType] - trainer: + required_hub_plugins: List[str] - parameters: + required_integrations: List[str] - gamma: 0.001 + requirements: Union[NoneType, str, List[str]] - Corresponds to: + skip_build: bool - from zenml import step, pipeline + source_files: SourceFileMode + + + target_repository: str + + + user: Optional[str] + + + resources: + + + cpu_count: Optional[PositiveFloat] + + + gpu_count: Optional[NonNegativeInt] + + + memory: Optional[ConstrainedStrValue] + + + step_operator: Optional[str] + + + success_hook_source: + + + attribute: Optional[str] + + + module: str + + + type: SourceType' + - 'as the ZenML quickstart. You can clone it like so:git clone --depth 1 git@github.com:zenml-io/zenml.git + + + cd zenml/examples/quickstart - @step + pip install -r requirements.txt - def trainer(gamma: float): + zenml init - # Use gamma as normal + To run a pipeline using the new stack: - print(gamma) + Set the stack as active on your clientCopyzenml stack set a_new_local_stack - @pipeline + Run your pipeline code:Copypython run.py --training-pipeline - def my_pipeline(gamma: float): + Keep this code handy as we''ll be using it in the next chapters! - # use gamma or pass it into the step + PreviousDeploying ZenML - print(0.01) + NextConnecting remote storage - trainer(gamma=gamma)' -- source_sentence: Can I use ZenML to register an S3 Artifact Store after setting - up the IAM user and generating the access key? + + Last updated 19 days ago' +- source_sentence: How do I register and connect an S3 artifact store in ZenML using + the interactive mode? sentences: - - ' to install and configure the AWS CLI on your hostyou don''t need to care about - enabling your other stack components (orchestrators, step operators, and model - deployers) to have access to the artifact store through IAM roles and policies + - 'hich Resource Name to use in the interactive mode:zenml artifact-store register + s3-zenfiles --flavor s3 --path=s3://zenfiles + + + zenml service-connector list-resources --resource-type s3-bucket --resource-id + s3://zenfiles + + + zenml artifact-store connect s3-zenfiles --connector aws-multi-type + + + Example Command Output + + + $ zenml artifact-store register s3-zenfiles --flavor s3 --path=s3://zenfiles + + + Running with active workspace: ''default'' (global) + + + Running with active stack: ''default'' (global) + + + Successfully registered artifact_store `s3-zenfiles`. - you can combine the S3 artifact store with other stack components that are not - running in AWS + $ zenml service-connector list-resources --resource-type s3-bucket --resource-id + zenfiles - Note: When you create the IAM user for your AWS access key, please remember to - grant the created IAM user permissions to read and write to your S3 bucket (i.e. - at a minimum: s3:PutObject, s3:GetObject, s3:ListBucket, s3:DeleteObject) + The ''s3-bucket'' resource with name ''zenfiles'' can be accessed by service + connectors configured in your workspace: - After having set up the IAM user and generated the access key, as described in - the AWS documentation, you can register the S3 Artifact Store as follows: + ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓ - # Store the AWS access key in a ZenML secret + ┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE + │ RESOURCE TYPE │ RESOURCE NAMES ┃ - zenml secret create s3_secret \ + ┠──────────────────────────────────────┼──────────────────────┼────────────────┼───────────────┼────────────────┨ - --aws_access_key_id='''' \ + ┃ 4a550c82-aa64-4a48-9c7f-d5e127d77a44 │ aws-multi-type │ 🔶 aws │ + 📦 s3-bucket │ s3://zenfiles ┃ - --aws_secret_access_key='''' + ┠──────────────────────────────────────┼──────────────────────┼────────────────┼───────────────┼────────────────┨ - # Register the S3 artifact-store and reference the ZenML secret + ┃ 66c0922d-db84-4e2c-9044-c13ce1611613 │ aws-multi-instance │ 🔶 aws │ + 📦 s3-bucket │ s3://zenfiles ┃ - zenml artifact-store register s3_store -f s3 \ + ┠──────────────────────────────────────┼──────────────────────┼────────────────┼───────────────┼────────────────┨ - --path=''s3://your-bucket'' \ + ┃ 65c82e59-cba0-4a01-b8f6-d75e8a1d0f55 │ aws-single-instance │ 🔶 aws │ + 📦 s3-bucket │ s3://zenfiles ┃ - --authentication_secret=s3_secret + ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛ - # Register and set a stack with the new artifact store + $ zenml artifact-store connect s3-zenfiles --connector aws-multi-type - zenml stack register custom_stack -a s3_store ... --set + Running with active workspace: ''default'' (global) - Advanced Configuration + Running with active stack: ''default'' (global) - The S3 Artifact Store accepts a range of advanced configuration options that can - be used to further customize how ZenML connects to the S3 storage service that - you are using. These are accessible via the client_kwargs, config_kwargs and s3_additional_kwargs - configuration attributes and are passed transparently to the underlying S3Fs library: + Successfully connected artifact store `s3-zenfiles` to the following resources:' + - '👣Step Operators - client_kwargs: arguments that will be transparently passed to the botocore client - . You can use it to configure parameters like endpoint_url and region_name when - connecting to an S3-compatible endpoint (e.g. Minio). + Executing individual steps in specialized environments. - config_kwargs: advanced parameters passed to botocore.client.Config. + The step operator enables the execution of individual pipeline steps in specialized + runtime environments that are optimized for certain workloads. These specialized + environments can give your steps access to resources like GPUs or distributed + processing frameworks like Spark. - s3_additional_kwargs: advanced parameters that are used when calling S3 API, typically - used for things like ServerSideEncryption and ACL.' - - 'Evidently + Comparison to orchestrators: The orchestrator is a mandatory stack component that + is responsible for executing all steps of a pipeline in the correct order and + providing additional features such as scheduling pipeline runs. The step operator + on the other hand is used to only execute individual steps of the pipeline in + a separate environment in case the environment provided by the orchestrator is + not feasible. - How to keep your data quality in check and guard against data and model drift - with Evidently profiling + When to use it + + + A step operator should be used if one or more steps of a pipeline require resources + that are not available in the runtime environments provided by the orchestrator. + An example would be a step that trains a computer vision model and requires a + GPU to run in a reasonable time, combined with a Kubeflow orchestrator running + on a Kubernetes cluster that does not contain any GPU nodes. In that case, it + makes sense to include a step operator like SageMaker, Vertex, or AzureML to execute + the training step with a GPU. + + + Step Operator Flavors + + + Step operators to execute steps on one of the big cloud providers are provided + by the following ZenML integrations: + + Step Operator Flavor Integration Notes SageMaker sagemaker aws Uses SageMaker + to execute steps Vertex vertex gcp Uses Vertex AI to execute steps AzureML azureml + azure Uses AzureML to execute steps Spark spark spark Uses Spark on Kubernetes + to execute steps in a distributed manner Custom Implementation custom Extend the + step operator abstraction and provide your own implementation - The Evidently Data Validator flavor provided with the ZenML integration uses Evidently - to perform data quality, data drift, model drift and model performance analyses, - to generate reports and run checks. The reports and check results can be used - to implement automated corrective actions in your pipelines or to render interactive - representations for further visual interpretation, evaluation and documentation. + If you would like to see the available flavors of step operators, you can use + the command: - When would you want to use it? + zenml step-operator flavor list - Evidently is an open-source library that you can use to monitor and debug machine - learning models by analyzing the data that they use through a powerful set of - data profiling and visualization features, or to run a variety of data and model - validation reports and tests, from data integrity tests that work with a single - dataset to model evaluation tests to data drift analysis and model performance - comparison tests. All this can be done with minimal configuration input from the - user, or customized with specialized conditions that the validation tests should - perform. + How to use it' + - 'Azure Container Registry - Evidently currently works with tabular data in pandas.DataFrame or CSV file formats - and can handle both regression and classification tasks. + Storing container images in Azure. - You should use the Evidently Data Validator when you need the following data and/or - model validation features that are possible with Evidently: + The Azure container registry is a container registry flavor that comes built-in + with ZenML and uses the Azure Container Registry to store container images. - Data Quality reports and tests: provides detailed feature statistics and a feature - behavior overview for a single dataset. It can also compare any two datasets. - E.g. you can use it to compare train and test data, reference and current data, - or two subgroups of one dataset. + + When to use it + + + You should use the Azure container registry if: - Data Drift reports and tests: helps detects and explore feature distribution changes - in the input data by comparing two datasets with identical schema.' - - ' us know! + one or more components of your stack need to pull or push container images. - Configuration at pipeline or step levelWhen running your ZenML pipeline with the - Sagemaker orchestrator, the configuration set when configuring the orchestrator - as a ZenML component will be used by default. However, it is possible to provide - additional configuration at the pipeline or step level. This allows you to run - whole pipelines or individual steps with alternative configurations. For example, - this allows you to run the training process with a heavier, GPU-enabled instance - type, while running other steps with lighter instances. + you have access to Azure. If you''re not using Azure, take a look at the other + container registry flavors. - Additional configuration for the Sagemaker orchestrator can be passed via SagemakerOrchestratorSettings. - Here, it is possible to configure processor_args, which is a dictionary of arguments - for the Processor. For available arguments, see the Sagemaker documentation . - Currently, it is not possible to provide custom configuration for the following - attributes: + How to deploy it + + Go here and choose a subscription, resource group, location, and registry name. + Then click on Review + Create and to create your container registry. - image_uri + How to find the registry URI - instance_count + The Azure container registry URI should have the following format: - sagemaker_session + .azurecr.io - entrypoint + # Examples: - base_job_name + zenmlregistry.azurecr.io - env + myregistry.azurecr.io - For example, settings can be provided in the following way: + To figure out the URI for your registry: - sagemaker_orchestrator_settings = SagemakerOrchestratorSettings( + Go to the Azure portal. - processor_args={ + In the search bar, enter container registries and select the container registry + you want to use. If you don''t have any container registries yet, check out the + deployment section on how to create one. - "instance_type": "ml.t3.medium", + Use the name of your registry to fill the template .azurecr.io + and get your URI. - "volume_size_in_gb": 30 + How to use it - They can then be applied to a step as follows: + To use the Azure container registry, we need: - @step(settings={"orchestrator.sagemaker": sagemaker_orchestrator_settings}) + Docker installed and running. - For example, if your ZenML component is configured to use ml.c5.xlarge with 400GB - additional storage by default, all steps will use it except for the step above, - which will use ml.t3.medium with 30GB additional storage. + The registry URI. Check out the previous section on the URI format and how to + get the URI for your registry. - Check out this docs page for more information on how to specify settings in general. + We can then register the container registry and use it in our active stack: - For more information and a full list of configurable attributes of the Sagemaker - orchestrator, check out the SDK Docs . + zenml container-registry register \ - S3 data access in ZenML steps' + + --flavor=azure \ + + + --uri= + + + # Add the container registry to the active stack + + + zenml stack update -c + + + You also need to set up authentication required to log in to the container registry. + + + Authentication Methods' model-index: - name: zenml/finetuned-all-MiniLM-L6-v2 results: @@ -1096,49 +1181,49 @@ model-index: type: dim_384 metrics: - type: cosine_accuracy@1 - value: 0.3313253012048193 + value: 0.3132530120481928 name: Cosine Accuracy@1 - type: cosine_accuracy@3 - value: 0.6024096385542169 + value: 0.6144578313253012 name: Cosine Accuracy@3 - type: cosine_accuracy@5 - value: 0.6927710843373494 + value: 0.7168674698795181 name: Cosine Accuracy@5 - type: cosine_accuracy@10 - value: 0.7710843373493976 + value: 0.7891566265060241 name: Cosine Accuracy@10 - type: cosine_precision@1 - value: 0.3313253012048193 + value: 0.3132530120481928 name: Cosine Precision@1 - type: cosine_precision@3 - value: 0.2008032128514056 + value: 0.20481927710843373 name: Cosine Precision@3 - type: cosine_precision@5 - value: 0.13855421686746985 + value: 0.1433734939759036 name: Cosine Precision@5 - type: cosine_precision@10 - value: 0.07710843373493974 + value: 0.0789156626506024 name: Cosine Precision@10 - type: cosine_recall@1 - value: 0.3313253012048193 + value: 0.3132530120481928 name: Cosine Recall@1 - type: cosine_recall@3 - value: 0.6024096385542169 + value: 0.6144578313253012 name: Cosine Recall@3 - type: cosine_recall@5 - value: 0.6927710843373494 + value: 0.7168674698795181 name: Cosine Recall@5 - type: cosine_recall@10 - value: 0.7710843373493976 + value: 0.7891566265060241 name: Cosine Recall@10 - type: cosine_ndcg@10 - value: 0.551850042031417 + value: 0.5579120329651274 name: Cosine Ndcg@10 - type: cosine_mrr@10 - value: 0.4812392426850258 + value: 0.48292933639319197 name: Cosine Mrr@10 - type: cosine_map@100 - value: 0.48905601510986996 + value: 0.4907452723782479 name: Cosine Map@100 - task: type: information-retrieval @@ -1148,49 +1233,49 @@ model-index: type: dim_256 metrics: - type: cosine_accuracy@1 - value: 0.3253012048192771 + value: 0.2891566265060241 name: Cosine Accuracy@1 - type: cosine_accuracy@3 - value: 0.5903614457831325 + value: 0.6144578313253012 name: Cosine Accuracy@3 - type: cosine_accuracy@5 - value: 0.6867469879518072 + value: 0.7108433734939759 name: Cosine Accuracy@5 - type: cosine_accuracy@10 - value: 0.7710843373493976 + value: 0.7650602409638554 name: Cosine Accuracy@10 - type: cosine_precision@1 - value: 0.3253012048192771 + value: 0.2891566265060241 name: Cosine Precision@1 - type: cosine_precision@3 - value: 0.19678714859437746 + value: 0.20481927710843373 name: Cosine Precision@3 - type: cosine_precision@5 - value: 0.13734939759036144 + value: 0.14216867469879516 name: Cosine Precision@5 - type: cosine_precision@10 - value: 0.07710843373493974 + value: 0.07650602409638553 name: Cosine Precision@10 - type: cosine_recall@1 - value: 0.3253012048192771 + value: 0.2891566265060241 name: Cosine Recall@1 - type: cosine_recall@3 - value: 0.5903614457831325 + value: 0.6144578313253012 name: Cosine Recall@3 - type: cosine_recall@5 - value: 0.6867469879518072 + value: 0.7108433734939759 name: Cosine Recall@5 - type: cosine_recall@10 - value: 0.7710843373493976 + value: 0.7650602409638554 name: Cosine Recall@10 - type: cosine_ndcg@10 - value: 0.5441628856415894 + value: 0.5394043126982406 name: Cosine Ndcg@10 - type: cosine_mrr@10 - value: 0.4714883342895392 + value: 0.46553595333715836 name: Cosine Mrr@10 - type: cosine_map@100 - value: 0.4790082728748276 + value: 0.4739275972429515 name: Cosine Map@100 - task: type: information-retrieval @@ -1200,49 +1285,49 @@ model-index: type: dim_128 metrics: - type: cosine_accuracy@1 - value: 0.3313253012048193 + value: 0.28313253012048195 name: Cosine Accuracy@1 - type: cosine_accuracy@3 - value: 0.5120481927710844 + value: 0.5481927710843374 name: Cosine Accuracy@3 - type: cosine_accuracy@5 - value: 0.6144578313253012 + value: 0.6506024096385542 name: Cosine Accuracy@5 - type: cosine_accuracy@10 - value: 0.6987951807228916 + value: 0.7168674698795181 name: Cosine Accuracy@10 - type: cosine_precision@1 - value: 0.3313253012048193 + value: 0.28313253012048195 name: Cosine Precision@1 - type: cosine_precision@3 - value: 0.1706827309236948 + value: 0.1827309236947791 name: Cosine Precision@3 - type: cosine_precision@5 - value: 0.12289156626506023 + value: 0.1301204819277108 name: Cosine Precision@5 - type: cosine_precision@10 - value: 0.06987951807228915 + value: 0.07168674698795179 name: Cosine Precision@10 - type: cosine_recall@1 - value: 0.3313253012048193 + value: 0.28313253012048195 name: Cosine Recall@1 - type: cosine_recall@3 - value: 0.5120481927710844 + value: 0.5481927710843374 name: Cosine Recall@3 - type: cosine_recall@5 - value: 0.6144578313253012 + value: 0.6506024096385542 name: Cosine Recall@5 - type: cosine_recall@10 - value: 0.6987951807228916 + value: 0.7168674698795181 name: Cosine Recall@10 - type: cosine_ndcg@10 - value: 0.5108893388836802 + value: 0.5067699591037801 name: Cosine Ndcg@10 - type: cosine_mrr@10 - value: 0.45126936316695354 + value: 0.43858529355517323 name: Cosine Mrr@10 - type: cosine_map@100 - value: 0.4605530012141939 + value: 0.44791284428498435 name: Cosine Map@100 - task: type: information-retrieval @@ -1252,49 +1337,49 @@ model-index: type: dim_64 metrics: - type: cosine_accuracy@1 - value: 0.2891566265060241 + value: 0.24096385542168675 name: Cosine Accuracy@1 - type: cosine_accuracy@3 - value: 0.4759036144578313 + value: 0.46987951807228917 name: Cosine Accuracy@3 - type: cosine_accuracy@5 - value: 0.5542168674698795 + value: 0.5843373493975904 name: Cosine Accuracy@5 - type: cosine_accuracy@10 - value: 0.6265060240963856 + value: 0.6807228915662651 name: Cosine Accuracy@10 - type: cosine_precision@1 - value: 0.2891566265060241 + value: 0.24096385542168675 name: Cosine Precision@1 - type: cosine_precision@3 - value: 0.15863453815261044 + value: 0.1566265060240964 name: Cosine Precision@3 - type: cosine_precision@5 - value: 0.1108433734939759 + value: 0.11686746987951806 name: Cosine Precision@5 - type: cosine_precision@10 - value: 0.06265060240963854 + value: 0.06807228915662648 name: Cosine Precision@10 - type: cosine_recall@1 - value: 0.2891566265060241 + value: 0.24096385542168675 name: Cosine Recall@1 - type: cosine_recall@3 - value: 0.4759036144578313 + value: 0.46987951807228917 name: Cosine Recall@3 - type: cosine_recall@5 - value: 0.5542168674698795 + value: 0.5843373493975904 name: Cosine Recall@5 - type: cosine_recall@10 - value: 0.6265060240963856 + value: 0.6807228915662651 name: Cosine Recall@10 - type: cosine_ndcg@10 - value: 0.45650145038804574 + value: 0.45307543718220417 name: Cosine Ndcg@10 - type: cosine_mrr@10 - value: 0.40227337923121054 + value: 0.3806679097341751 name: Cosine Mrr@10 - type: cosine_map@100 - value: 0.4137670603435629 + value: 0.389050349953244 name: Cosine Map@100 --- @@ -1348,9 +1433,9 @@ from sentence_transformers import SentenceTransformer model = SentenceTransformer("zenml/finetuned-all-MiniLM-L6-v2") # Run inference sentences = [ - 'Can I use ZenML to register an S3 Artifact Store after setting up the IAM user and generating the access key?', - " to install and configure the AWS CLI on your hostyou don't need to care about enabling your other stack components (orchestrators, step operators, and model deployers) to have access to the artifact store through IAM roles and policies\n\nyou can combine the S3 artifact store with other stack components that are not running in AWS\n\nNote: When you create the IAM user for your AWS access key, please remember to grant the created IAM user permissions to read and write to your S3 bucket (i.e. at a minimum: s3:PutObject, s3:GetObject, s3:ListBucket, s3:DeleteObject)\n\nAfter having set up the IAM user and generated the access key, as described in the AWS documentation, you can register the S3 Artifact Store as follows:\n\n# Store the AWS access key in a ZenML secret\n\nzenml secret create s3_secret \\\n\n--aws_access_key_id='' \\\n\n--aws_secret_access_key=''\n\n# Register the S3 artifact-store and reference the ZenML secret\n\nzenml artifact-store register s3_store -f s3 \\\n\n--path='s3://your-bucket' \\\n\n--authentication_secret=s3_secret\n\n# Register and set a stack with the new artifact store\n\nzenml stack register custom_stack -a s3_store ... --set\n\nAdvanced Configuration\n\nThe S3 Artifact Store accepts a range of advanced configuration options that can be used to further customize how ZenML connects to the S3 storage service that you are using. These are accessible via the client_kwargs, config_kwargs and s3_additional_kwargs configuration attributes and are passed transparently to the underlying S3Fs library:\n\nclient_kwargs: arguments that will be transparently passed to the botocore client . You can use it to configure parameters like endpoint_url and region_name when connecting to an S3-compatible endpoint (e.g. Minio).\n\nconfig_kwargs: advanced parameters passed to botocore.client.Config.\n\ns3_additional_kwargs: advanced parameters that are used when calling S3 API, typically used for things like ServerSideEncryption and ACL.", - ' us know!\n\nConfiguration at pipeline or step levelWhen running your ZenML pipeline with the Sagemaker orchestrator, the configuration set when configuring the orchestrator as a ZenML component will be used by default. However, it is possible to provide additional configuration at the pipeline or step level. This allows you to run whole pipelines or individual steps with alternative configurations. For example, this allows you to run the training process with a heavier, GPU-enabled instance type, while running other steps with lighter instances.\n\nAdditional configuration for the Sagemaker orchestrator can be passed via SagemakerOrchestratorSettings. Here, it is possible to configure processor_args, which is a dictionary of arguments for the Processor. For available arguments, see the Sagemaker documentation . Currently, it is not possible to provide custom configuration for the following attributes:\n\nimage_uri\n\ninstance_count\n\nsagemaker_session\n\nentrypoint\n\nbase_job_name\n\nenv\n\nFor example, settings can be provided in the following way:\n\nsagemaker_orchestrator_settings = SagemakerOrchestratorSettings(\n\nprocessor_args={\n\n"instance_type": "ml.t3.medium",\n\n"volume_size_in_gb": 30\n\nThey can then be applied to a step as follows:\n\n@step(settings={"orchestrator.sagemaker": sagemaker_orchestrator_settings})\n\nFor example, if your ZenML component is configured to use ml.c5.xlarge with 400GB additional storage by default, all steps will use it except for the step above, which will use ml.t3.medium with 30GB additional storage.\n\nCheck out this docs page for more information on how to specify settings in general.\n\nFor more information and a full list of configurable attributes of the Sagemaker orchestrator, check out the SDK Docs .\n\nS3 data access in ZenML steps', + 'How do I register and connect an S3 artifact store in ZenML using the interactive mode?', + "hich Resource Name to use in the interactive mode:zenml artifact-store register s3-zenfiles --flavor s3 --path=s3://zenfiles\n\nzenml service-connector list-resources --resource-type s3-bucket --resource-id s3://zenfiles\n\nzenml artifact-store connect s3-zenfiles --connector aws-multi-type\n\nExample Command Output\n\n$ zenml artifact-store register s3-zenfiles --flavor s3 --path=s3://zenfiles\n\nRunning with active workspace: 'default' (global)\n\nRunning with active stack: 'default' (global)\n\nSuccessfully registered artifact_store `s3-zenfiles`.\n\n$ zenml service-connector list-resources --resource-type s3-bucket --resource-id zenfiles\n\nThe 's3-bucket' resource with name 'zenfiles' can be accessed by service connectors configured in your workspace:\n\n┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓\n\n┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE │ RESOURCE TYPE │ RESOURCE NAMES ┃\n\n┠──────────────────────────────────────┼──────────────────────┼────────────────┼───────────────┼────────────────┨\n\n┃ 4a550c82-aa64-4a48-9c7f-d5e127d77a44 │ aws-multi-type │ 🔶 aws │ 📦 s3-bucket │ s3://zenfiles ┃\n\n┠──────────────────────────────────────┼──────────────────────┼────────────────┼───────────────┼────────────────┨\n\n┃ 66c0922d-db84-4e2c-9044-c13ce1611613 │ aws-multi-instance │ 🔶 aws │ 📦 s3-bucket │ s3://zenfiles ┃\n\n┠──────────────────────────────────────┼──────────────────────┼────────────────┼───────────────┼────────────────┨\n\n┃ 65c82e59-cba0-4a01-b8f6-d75e8a1d0f55 │ aws-single-instance │ 🔶 aws │ 📦 s3-bucket │ s3://zenfiles ┃\n\n┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛\n\n$ zenml artifact-store connect s3-zenfiles --connector aws-multi-type\n\nRunning with active workspace: 'default' (global)\n\nRunning with active stack: 'default' (global)\n\nSuccessfully connected artifact store `s3-zenfiles` to the following resources:", + "Azure Container Registry\n\nStoring container images in Azure.\n\nThe Azure container registry is a container registry flavor that comes built-in with ZenML and uses the Azure Container Registry to store container images.\n\nWhen to use it\n\nYou should use the Azure container registry if:\n\none or more components of your stack need to pull or push container images.\n\nyou have access to Azure. If you're not using Azure, take a look at the other container registry flavors.\n\nHow to deploy it\n\nGo here and choose a subscription, resource group, location, and registry name. Then click on Review + Create and to create your container registry.\n\nHow to find the registry URI\n\nThe Azure container registry URI should have the following format:\n\n.azurecr.io\n\n# Examples:\n\nzenmlregistry.azurecr.io\n\nmyregistry.azurecr.io\n\nTo figure out the URI for your registry:\n\nGo to the Azure portal.\n\nIn the search bar, enter container registries and select the container registry you want to use. If you don't have any container registries yet, check out the deployment section on how to create one.\n\nUse the name of your registry to fill the template .azurecr.io and get your URI.\n\nHow to use it\n\nTo use the Azure container registry, we need:\n\nDocker installed and running.\n\nThe registry URI. Check out the previous section on the URI format and how to get the URI for your registry.\n\nWe can then register the container registry and use it in our active stack:\n\nzenml container-registry register \\\n\n--flavor=azure \\\n\n--uri=\n\n# Add the container registry to the active stack\n\nzenml stack update -c \n\nYou also need to set up authentication required to log in to the container registry.\n\nAuthentication Methods", ] embeddings = model.encode(sentences) print(embeddings.shape) @@ -1396,43 +1481,43 @@ You can finetune this model on your own dataset. | Metric | Value | |:--------------------|:-----------| -| cosine_accuracy@1 | 0.3313 | -| cosine_accuracy@3 | 0.6024 | -| cosine_accuracy@5 | 0.6928 | -| cosine_accuracy@10 | 0.7711 | -| cosine_precision@1 | 0.3313 | -| cosine_precision@3 | 0.2008 | -| cosine_precision@5 | 0.1386 | -| cosine_precision@10 | 0.0771 | -| cosine_recall@1 | 0.3313 | -| cosine_recall@3 | 0.6024 | -| cosine_recall@5 | 0.6928 | -| cosine_recall@10 | 0.7711 | -| cosine_ndcg@10 | 0.5519 | -| cosine_mrr@10 | 0.4812 | -| **cosine_map@100** | **0.4891** | +| cosine_accuracy@1 | 0.3133 | +| cosine_accuracy@3 | 0.6145 | +| cosine_accuracy@5 | 0.7169 | +| cosine_accuracy@10 | 0.7892 | +| cosine_precision@1 | 0.3133 | +| cosine_precision@3 | 0.2048 | +| cosine_precision@5 | 0.1434 | +| cosine_precision@10 | 0.0789 | +| cosine_recall@1 | 0.3133 | +| cosine_recall@3 | 0.6145 | +| cosine_recall@5 | 0.7169 | +| cosine_recall@10 | 0.7892 | +| cosine_ndcg@10 | 0.5579 | +| cosine_mrr@10 | 0.4829 | +| **cosine_map@100** | **0.4907** | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) -| Metric | Value | -|:--------------------|:----------| -| cosine_accuracy@1 | 0.3253 | -| cosine_accuracy@3 | 0.5904 | -| cosine_accuracy@5 | 0.6867 | -| cosine_accuracy@10 | 0.7711 | -| cosine_precision@1 | 0.3253 | -| cosine_precision@3 | 0.1968 | -| cosine_precision@5 | 0.1373 | -| cosine_precision@10 | 0.0771 | -| cosine_recall@1 | 0.3253 | -| cosine_recall@3 | 0.5904 | -| cosine_recall@5 | 0.6867 | -| cosine_recall@10 | 0.7711 | -| cosine_ndcg@10 | 0.5442 | -| cosine_mrr@10 | 0.4715 | -| **cosine_map@100** | **0.479** | +| Metric | Value | +|:--------------------|:-----------| +| cosine_accuracy@1 | 0.2892 | +| cosine_accuracy@3 | 0.6145 | +| cosine_accuracy@5 | 0.7108 | +| cosine_accuracy@10 | 0.7651 | +| cosine_precision@1 | 0.2892 | +| cosine_precision@3 | 0.2048 | +| cosine_precision@5 | 0.1422 | +| cosine_precision@10 | 0.0765 | +| cosine_recall@1 | 0.2892 | +| cosine_recall@3 | 0.6145 | +| cosine_recall@5 | 0.7108 | +| cosine_recall@10 | 0.7651 | +| cosine_ndcg@10 | 0.5394 | +| cosine_mrr@10 | 0.4655 | +| **cosine_map@100** | **0.4739** | #### Information Retrieval * Dataset: `dim_128` @@ -1440,21 +1525,21 @@ You can finetune this model on your own dataset. | Metric | Value | |:--------------------|:-----------| -| cosine_accuracy@1 | 0.3313 | -| cosine_accuracy@3 | 0.512 | -| cosine_accuracy@5 | 0.6145 | -| cosine_accuracy@10 | 0.6988 | -| cosine_precision@1 | 0.3313 | -| cosine_precision@3 | 0.1707 | -| cosine_precision@5 | 0.1229 | -| cosine_precision@10 | 0.0699 | -| cosine_recall@1 | 0.3313 | -| cosine_recall@3 | 0.512 | -| cosine_recall@5 | 0.6145 | -| cosine_recall@10 | 0.6988 | -| cosine_ndcg@10 | 0.5109 | -| cosine_mrr@10 | 0.4513 | -| **cosine_map@100** | **0.4606** | +| cosine_accuracy@1 | 0.2831 | +| cosine_accuracy@3 | 0.5482 | +| cosine_accuracy@5 | 0.6506 | +| cosine_accuracy@10 | 0.7169 | +| cosine_precision@1 | 0.2831 | +| cosine_precision@3 | 0.1827 | +| cosine_precision@5 | 0.1301 | +| cosine_precision@10 | 0.0717 | +| cosine_recall@1 | 0.2831 | +| cosine_recall@3 | 0.5482 | +| cosine_recall@5 | 0.6506 | +| cosine_recall@10 | 0.7169 | +| cosine_ndcg@10 | 0.5068 | +| cosine_mrr@10 | 0.4386 | +| **cosine_map@100** | **0.4479** | #### Information Retrieval * Dataset: `dim_64` @@ -1462,21 +1547,21 @@ You can finetune this model on your own dataset. | Metric | Value | |:--------------------|:-----------| -| cosine_accuracy@1 | 0.2892 | -| cosine_accuracy@3 | 0.4759 | -| cosine_accuracy@5 | 0.5542 | -| cosine_accuracy@10 | 0.6265 | -| cosine_precision@1 | 0.2892 | -| cosine_precision@3 | 0.1586 | -| cosine_precision@5 | 0.1108 | -| cosine_precision@10 | 0.0627 | -| cosine_recall@1 | 0.2892 | -| cosine_recall@3 | 0.4759 | -| cosine_recall@5 | 0.5542 | -| cosine_recall@10 | 0.6265 | -| cosine_ndcg@10 | 0.4565 | -| cosine_mrr@10 | 0.4023 | -| **cosine_map@100** | **0.4138** | +| cosine_accuracy@1 | 0.241 | +| cosine_accuracy@3 | 0.4699 | +| cosine_accuracy@5 | 0.5843 | +| cosine_accuracy@10 | 0.6807 | +| cosine_precision@1 | 0.241 | +| cosine_precision@3 | 0.1566 | +| cosine_precision@5 | 0.1169 | +| cosine_precision@10 | 0.0681 | +| cosine_recall@1 | 0.241 | +| cosine_recall@3 | 0.4699 | +| cosine_recall@5 | 0.5843 | +| cosine_recall@10 | 0.6807 | +| cosine_ndcg@10 | 0.4531 | +| cosine_mrr@10 | 0.3807 | +| **cosine_map@100** | **0.3891** |