The following figure shows the typical I/O software stack that is needed to make optimum use of the storage hardware in an HPC system. The role of the High Level I/O Library is often crucial. First, it conveniently maps application abstractions to a structured, portable file format. Second, when the library is a parallel I/O library, it can achieve superior I/O rates by going through the MPI-IO interface to access an underlying parallel file system like Lustre, as described in the Parallel I/O roadmap.

In a parallel I/O software stack, the high level I/O library sits between the application and the MPI-IO layer. The MPI-IO layer interfaces with the parallel file system, which interfaces with the storage hardware.
PnetCDF, PHDF5, and ADIOS are examples of parallel high level I/O libraries.

The present roadmap includes topics about two major high-level libraries: PnetCDF, a parallel implementation of the netCDF API and the related file format; and PHDF5, a parallel library for working with HDF5 formatted files. ADIOS is also touched upon briefly.

 
©  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Inclusivity Statement