GPU Memory
Steve Lantz
Cornell Center for Advanced Computing
Revisions: 5/2023, 12/2021, 5/2021 (original)
Just like a CPU, the GPU relies on a memory hierarchy—from RAM, through cache levels—to ensure that its processing engines are kept supplied with the data they need to do useful work. And just like the cores in a CPU, the streaming multiprocessors (SMs) in a GPU ultimately require the data to be in registers to be available for computations. This topic looks at the sizes and properties of the different elements of the GPU's memory hierarchy and how they compare to those found in CPUs.
Objectives
After you complete this topic, you should be able to:
- List the main features of GPU memory and explain how they differ from comparable features of CPUs
- Describe the names, sizes, and speeds of the memory components of specific models of NVIDIA GPU devices
Prerequisites
- Familiarity with High Performance Computing (HPC) concepts could be helpful, but most terms are explained in context.
- Parallel Programming Concepts and High-Performance Computing could be considered as a possible companion to this topic, for those who seek to expand their knowledge of parallel computing in general, as well as on GPUs.