Thinking Big
Christopher J. Cameron, Steve Lantz
Cornell Center for Advanced Computing
Revisions: 3/2022, 9/2014 (original)
This topic identifies special considerations and strategies for scaling code to run on supercomputers. Scaling up used to mean using 8, 16, or 32 cores, but now scaling up means hundreds, thousands, or even hundreds of thousands of cores. This requires different thinking because many programs that scale acceptably on smaller machines will not perform well when scaled to large machines.
Objectives
After you complete this topic, you should be able to:
- Explain why parallelizing the outermost loop may not be the best strategy
- Explain how the domain decomposition choice impacts communication among tasks
- Demonstrate dividing data into compact domains
- List the four steps of Foster's methodology for program design
- Identify three types of program activity that might benefit from optimization
Prerequisites
This topic assumes a basic understanding of serial (single-threaded) programming and familiarity with computer terminology.