Skip to main content


Introduction

Many scientific applications work with structured data and in many cases such data requires pre- and post- processing. The existing I/O libraries not only allow applications to work with portable, self-describing file formats, but also provide tools to process the data. In this module, we introduce parallel I/O libraries and techniques that can be used to increase throughput and efficiency of I/O bound applications. Efficient parallel I/O becomes extremely important as we scale scientific applications across large number of nodes comprising of multicores and accelerators. This module is the first part of a two-part parallel I/O tutorial and will focus on the use of the parallel NetCDF I/O library. The planned second part of this tutorial will focus on PHDF5 and give a brief introduction to ADIOS. A separate module covers Parallel I/O Basics.

Manu Shantharam
San Diego Supercomputer Center
July 2017