# Installation This page explains how to install the library. We assume that you have a basic understanding of the Python programming language and are familiar with machine learning models. Using [Docker](https://www.docker.com/) is optional but not required. You should be comfortable with the Python programming language and know how to work with Machine Learning models. Using [Docker](https://www.docker.com/) is optional and not necessary. The library can be installed in any of the following ways: - system environment - virtualenv - [Docker](https://www.docker.com/) image If you opt for using Docker, you can get NVIDIA optimized Docker images for Python frameworks from the [NVIDIA NGC Catalog](https://catalog.ngc.nvidia.com/containers). To run model inference on NVIDIA GPU using the Docker runtime, we recommend that you install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/overview.html), which enables GPU acceleration for containers. ## Prerequisites Before installing the library, ensure that you meet the following requirements: - An operating system with glibc >= `2.35`. Triton Inference Server and PyTriton have only been rigorously tested on Ubuntu 22.04. Other supported operating systems include Ubuntu Debian 11+, Rocky Linux 9+, and Red Hat Universal Base Image 9+. - to check your glibc version, run `ldd --version` - Python version >= `3.8` - `pip >= `20.3` - `libpython3.*.so` available in the operating system (appropriate for Python version). ## Install from `pypi` You can install the package from [pypi.org](https://pypi.org/project/nvidia-pytriton/) by running the following command: ```shell pip install -U nvidia-pytriton ``` !!! note "Triton Inference Server binaries" The Triton Inference Server binaries are installed as part of the PyTriton package. ## Setting Up Python Environment The Triton Inference Server is automatically run with your Python interpreter version. To use Triton binary you need to make sure that `libpython3.*.so` library can be linked during PyTriton start. Install and provide location to `libpython3.*.so` library in LD_LIBRARY_PATH before you will run PyTriton. Below we presented some options on how to prepare your Python environment to run PyTriton with common tools. ### Upgrading `pip` version You need to have `pip` version 20.3 or higher. To upgrade an older version of pip, run this command: ```shell pip install -U pip ``` ### Using system interpreter When you are running PyTriton on Ubuntu 22.04 install the desired Python interpreter and `libpython3*so.` library. ```shell # Install necessary packages apt update -y apt install -y software-properties-common # Add repository with various Python versions add-apt-repository ppa:deadsnakes/ppa -y # Install Python 3.8 apt install -y python3.8 libpython3.8 python3.8-distutils python3-pip \ build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev \ libffi-dev curl libbz2-dev pkg-config make # Install library for interpreter python3.8 -m pip install nvidia-pytriton ``` ### Creating virtualenv using `pyenv` In order to install different version replace the `3.8` with desired Python version in the example below: ```shell # Install necessary packages apt update -y apt install -y python3 python3-distutils python-is-python3 git \ build-essential libssl-dev zlib1g-dev \ libbz2-dev libreadline-dev libsqlite3-dev curl \ libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev # Install pyenv curl https://pyenv.run | bash # Configure pyenv in current environment export PYENV_ROOT="$HOME/.pyenv" command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH" eval "$(pyenv init -)" eval "$(pyenv virtualenv-init -)" # Install Python 3.8 with shared library support env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.8 # Create and activate virtualenv pyenv virtualenv 3.8 venv pyenv activate venv # export the library path export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pyenv virtualenv-prefix)/lib # Install library for interpreter pip install nvidia-pytriton ``` ### Creating virtualenv using `venv` In order to install different version replace the `3.8` with desired Python version in the example below: ```shell # Install necessary packages apt update -y apt install -y software-properties-common # Add repository with various Python versions add-apt-repository ppa:deadsnakes/ppa -y # Install Python 3.8 apt install -y python3.8 libpython3.8 python3.8-distutils python3.8-venv python3.8-pip python-is-python3 \ build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev \ libffi-dev curl libbz2-dev pkg-config make # Create and activate virtualenv python3.8 -m venv /opt/venv source /opt/venv/bin/activate # Install library for interpreter pip install nvidia-pytriton ``` ### Creating virtualenv using `miniconda` In order to install different version replace the `3.8` with desired Python version in the example below: ```shell # Install necessary packages apt update -y apt install -y python3 python3-distutils python-is-python3 curl # Download miniconda CONDA_VERSION=latest TARGET_MACHINE=x86_64 curl "https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-${TARGET_MACHINE}.sh" --output miniconda.sh # Install miniconda and add to PATH bash miniconda.sh export PATH=~/miniconda3/bin/:$PATH # Initialize bash conda init bash bash # Create and activate virtualenv conda create -c conda-forge -n venv python=3.8 conda activate venv # Export the library path export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib # Install library for interpreter pip install nvidia-pytriton ``` ## Building binaries from source The binary package can be built from the source, allowing access to unreleased hotfixes, the ability to modify the PyTriton code, and compatibility with various Triton Inference Server versions, including custom server builds. For further information on building the PyTriton binary, refer to the [Building](building.md) page.