Machine Learning on Horizon

The Vista User Guide, in its Machine Learning section, has guidance that applies to Horizon as well. Vista offers detailed guides on using PyTorch as well as Transformers and Accelerate from Hugging Face. In both cases, training can be run on one single Grace Blackwell compute node or scaled across multiple GB nodes on Horizon.

• Running PyTorch on Vista
• Setting Up Hugging Face Transformers and Accelerate on Vista

Of course, you are free to install and run ML packages other than these two, according to your own needs and preferences. To take full advantange of Horizon, though, you will need to understand how to set up your software such that it can use the GPUs on one or multiple GH nodes.

Note that when you install Python packages that are to be used across multiple compute nodes, it should be done in a virtual environment that you have created under the $SCRATCH directory rather than your $HOME directory. This prevents undue stress on the file system containing $HOME.