In small clusters with problems of modest size, nearly all of the time is spent in computation. With computer systems like Stampede2 that have thousands of nodes, the computation may not be the only time-consuming part of the job. Attention also needs to be paid to I/O (both input and output) and the startup and shutdown of the job.

Ultimately I/O needs to be parallelized too, and some strategies for doing this are discussed in the Parallel I/O topics (Parallel I/O and Parallel I/O Libraries Part 1 & Part 2 ). Furthermore, systems programmers are constantly working to decrease startup times for MPI and other software infrastructure. Issues like these are bound to assume greater importance in the exascale era.