Deploying and Connecting to Google TPU Instances via GCloud CLI

Context

We assume the reader has already created a Google Cloud Platform (GCP) user or organization account and an associated project.

We also assume the reader to have the Google Cloud CLI installed. If not, please follow the links right after to install and setup.

Creating the initial TPU VM on GCP

In order to create your initial TPU instance, you will need to provide some information:

• The GCP zone you would like to see the instance being deployed (close to the reader for development purposes, close to the end user for production, for instance)
• Which kind of TPU you would like to target
• Which version of the TPU runtime you want to leverage on the instance
• Custom instance name to quickly skim and refer back to the instance

Overall, the end command looks like this:

gcloud compute tpus tpu-vm create <ref_instance_name> \
--zone=<deploiment_zone> \
--accelerator-type=<target_tpu_generation> \
--version=<runtime_version>

Deploying a TPU v5litepod-8 instance

In our case, we will be deploying a v5litepod-8 instance name optimum-tpu-get-started in the GCP region us-west4-a using the latest v2-alpha-tpuv5-lite runtime version.

Of course, feel free to adjust all these parameters to the one that match with your usage and quotas.

Before creating the instance, please make sure to install gcloud alpha component as it is required to be able to target TPUv5 VMs: gcloud components install alpha

gcloud alpha compute tpus tpu-vm create optimum-tpu-get-started \
--zone=us-west4-a \
--accelerator-type=v5litepod-8 \
--version=v2-alpha-tpuv5-lite

Connecting to the instance via ssh

gcloud compute tpus tpu-vm ssh <ref_instance_name> --zone=<deployment_zone>
$ >

In the example above deploying v5litepod-8 it would be something like:

gcloud compute tpus tpu-vm ssh optimum-tpu-get-started --zone=us-west4-a
$ >

Other useful commands

This is used to get information about the tpu-vm for example its external IP:

gcloud compute tpus tpu-vm describe --zone=<tpu_zone> <tpu_name>

Next steps

• If you wish to train your own model, you can now install optimum-tpu
• If you wish do to serving, you can look at our serving tutorial