Memory, Cache, Interconnects
Chris Myers, Steve Lantz
Cornell Center for Advanced Computing
Revisions: 4/2024, 3/2023, 1/2022, 2/2021 (original)
In all modern processors, good use of memory hierarchies is crucial in getting good performance. Hierarchical memory—from RAM on multiple nodes, through successive levels of cache—is used to feed data down to a single core's hardware registers for computations, and then back out for storage in memory. See some of our companion material on code optimization and memory hierarchy for some more detailed background on this subject, as well as some further specifics about the structure of memory and cache in the Skylake SKX nodes on Stampede3.
Objectives
After you complete this topic, you should be able to:
- Explain the importance of memory hierarchy in computations
- Convert SKX memory or cache bandwidth numbers to 512-bit vectors per cycle
- Explain why good cache utilization is so important in achieving peak performance
- Define last level cache and its role in data movement
- Identify the main role of network interconnects in advanced clusters
- Describe how network traffic moves among nodes on Stampede2 and Frontera
- Define the term "fat tree topology"
Prerequisites
- Familiarity with High Performance Computing (HPC) concepts. Those who are less conversant with HPC terms and techniques should be prepared to inspect the glossary terms rather frequently. It may also be helpful to review Cornell Virtual Workshop content on Parallel Programming Concepts and High-Performance Computing and either MPI or OpenMP.
- Programming experience in C or Fortran. Introductions to C and Fortran are available, though the reader will need to look elsewhere for a full tutorial on these languages.
- Readers who need an introduction to either Stampede3 or Frontera will find it helpful to first review one of more of the following items: the Stampede3 User Guide, the Frontera User Guide, and the Getting Started on Frontera CVW material.