Parallel I/O Basics
Steve Lantz
Cornell Center for Advanced Computing
Revisions: 10/2022, 5/2017, 6/2015, 2/2014, 7/2012 (original)
Parallel I/O involves coordinated and optimized data movement. Modern HPC systems offer software and interfaces to make overcoming these challenges easier. This topic describes the components of a modern parallel I/O stack and identifies specific software libraries available on HPC systems.
Objectives
After you complete this topic, you should be able to:
- Explain the layers of a parallel I/O stack
- List alternatives to MPI-IO for parallel I/O
Prerequisites
The Parallel I/O roadmap assumes that the reader has basic knowledge of Linux shell commands, parallel programming, and MPI. Coverage of these prerequisites can be found in the Shells topic plus the roadmaps on Parallel Programming Concepts and High-Performance Computing and MPI Basics.
Programming experience in C or Fortran is also recommended. Introductory roadmaps on C and Fortran are available, though the reader will need to look elsewhere for a full tutorial on these languages.
In sequence, the current roadmap logically follows the MPI Advanced Topics roadmap, but the latter is not a prerequisite.