To compile code with OpenMP offloading to NVIDIA GPUs, you need to specify the following compiler-specific flags, as recommended by TACC:

When compiling with the NVIDIA HPC compilers, it is recommended to specify the target architecture with -gpu=cc100 on Horizon or -gpu=cc90 on Vista. If this option is omitted and the target GPU is not present at compile time (e.g., on a login node), your code may instead be compiled for a broad range of compute capabilities, producing a larger executable. Specifying the Blackwell architecture that is found on Horizon, or the Hopper architecture that is found on Vista, keeps the build smaller. But the NVIDIA environment module that is loaded by default may not support the most recent GPU architectures, so you may have to load a module for a later release, e.g., module load nvidia/26.1 on Vista.

For GCC, the above flags only work if NVIDIA GPU support was enabled at the time that GCC was installed. You can check this with the command gcc -v 2>&1 | grep "offload". If NVIDIA support was enabled, you should see --enable-offload-targets=nvptx-none included in the output. At TACC, this is true for all GCC versions on Horizon and Vista, but not on older clusters.

OMP_TARGET_OFFLOAD

The OMP_TARGET_OFFLOAD environment variable controls what happens when a target region is encountered at runtime. It accepts three values: default, disabled, and mandatory. The default behavior of OpenMP is to offload to the device if one is available, and fall back to the host otherwise. This behavior can be explicitly defined with OMP_TARGET_OFFLOAD=default. If it is set to disabled, it runs the target region on the host, and if it is set to mandatory, it aborts the program if offloading is not possible.