Python is the most popular programming language for both ML and general-purpose programming, and it's no surprise that a lot of container images are built around this fact. The NVIDIA NGC Catalog and Docker Hub maintain a list of images that are widely used by HPC users. These images will provide you with a complete, consistent Python environment, with the right configuration for GPU workloads. In addition, the convenience of not having to install individual Python packages is one of the greatest advantages of using these containers.

When using these containers, you may sometimes want to install additional packages to supplement what is already in the container. This can be done by using the package manager pip inside the container. On Vista, when pip is run inside a container, packages will be installed in your user space at $HOME/.local, which is the default location for user-installed Python packages. As long as the home directory is bound to the container (which is the default at TACC) and the user-installed packages are in sys.path, the installed packages will be available for use within the container.

However, this could be a point of error if you manage multiple containers of different Python versions that all use the same user-installed packages. If the version of a package installed by one container is incompatible with the requirements of another container, an error would occur. In this case, it is recommended to create virtual environments with the --system-site-packages option in each container, which will allow the container to use the packages that came with the container while including your user-installed packages in the virtual environment.

Use Containers as Kernels in JupyterLab

In JupyterLab, notebooks use the default version of Python installed on the host system as the kernel, i.e., as the Python interpreter that runs the code in the notebooks. Alternatively, you may use any container as the kernel for notebooks, as long as the container image is accessible and the json that defines the kernel, $HOME/.local/share/jupyter/kernels/<kernel_name>/kernel.json, includes the following:

Then, when you launch a Jupyter notebook, you may switch the kernel to <kernel_name> to use the container as the kernel. One advantage of doing so is that the container may come with a pre-installed suite of packages that are automatically incorporated into the kernel's environment.