Many scientific applications work with structured data, and in many cases such data require pre- and post-processing. I/O libraries exist that not only allow applications to work with portable, self-describing file formats, but also provide tools to process the data. In this roadmap, we introduce parallel I/O libraries and techniques that can be used to increase the throughput and efficiency of I/O bound applications. Efficient parallel I/O becomes extremely important as we scale scientific applications across large numbers of nodes comprised of multiple cores and accelerators.


After you complete this roadmap, you should be able to:

  • Summarize the motivation for using I/O libraries like netCDF and HDF5, as well as their parallel counterparts PnetCDF and PHDF5.
  • Describe the basic use of the PnetCDF and PHDF5 APIs, as well as the rationale for the ADIOS API.
  • Use a high level I/O library with parallel code.

The examples and exercises in this roadmap are designed to run on Stampede2 or Frontera. To use these systems, you need:

  • A TACC account to login to Stampede2 or Frontera
  • A compute time allocation for Stampede2 or Frontera
©   Cornell University  |  Center for Advanced Computing  |  Copyright Statement  |  Inclusivity Statement