Creating Hybrid Configurations
Steve Lantz (original author), Brandon Barker
Cornell Center for Advanced Computing
Revisions: 2/2024, 1/2023, 11/2020, 1/2014, 6/2011 (original)
On heterogeneous NUMA systems like Frontera and Stampede3, placement and binding of processes and their associated memory are important performance considerations. Process Affinity and Memory Policy have a significant effect on pure MPI, pure OpenMP, and hybrid codes. Simple numactl commands, APIs, and environment variables let users control affinity of processes, threads, and memory allocations.
Objectives
After you complete this workshop, you should be able to:
- Explain the role of process affinity and memory policy in achieving different MPI and thread configurations
- Explain the impact of numactl at the process level
- Identify different methods for configuring process affinity and memory policy
- Demonstrate preparing a hybrid batch script and program templates
Prerequisites
- A working knowledge of general programming concepts
- A working knowledge of Linux; otherwise, try working through the Linux topic first
- Ability to program in a high-level language such as Fortran or C
- A basic familiarity with parallel programming concepts
- A basic working knowledge of MPI at the level of the Cornell Virtual Workshop roadmap on the Message Passing Interface (MPI)
- A basic working knowledge of OpenMP at the level of the Cornell Virtual Workshop roadmap on OpenMP