Roadmap: Parallel I/O
This roadmap presents basic concepts and techniques that will allow your application to take advantage of parallel I/O to increase throughput and improve scalability. The parallel I/O software stack is introduced from the hardware level on up. Emphasis is placed on the Lustre parallel file system, and on MPI-IO as a fundamental API for enabling parallel I/O. These are the building blocks of typical HPC software stacks, including those available on the HPC systems at TACC.
Objectives
After you complete this roadmap, you should be able to:
- Demonstrate using MPI-IO (or a library built on top of it) as the interface to a parallel file system such as Lustre
- Explain how MPI-IO interoperates with the file system to enhance I/O performance for distributed-memory applications, especially those with heavy file-I/O demands
Prerequisites
This roadmap assumes that the reader has basic knowledge of Linux shell commands, parallel programming, and MPI. Coverage of these prerequisites can be found in the Shells topic plus the roadmaps on Parallel Programming Concepts and High-Performance Computing and MPI Basics.
Programming experience in C or Fortran is also recommended. Introductory roadmaps on C and Fortran are available, though the reader will need to look elsewhere for a full tutorial on these languages.
In sequence, this roadmap logically follows the MPI Advanced Topics roadmap, but the latter is not a prerequisite.