Lustre
Steve Lantz
Cornell Center for Advanced Computing
Revisions: 10/2022, 5/2017, 6/2015, 2/2014, 7/2012 (original)
Lustre is a parallel distributed file system that is often featured in high performance computing environments. The Lustre file system, because it is scalable by design, can support many thousands of client systems accessing tens of petabytes of storage, at an aggregate rate of hundreds of gigabytes per second (GB/s).
Objectives
After you complete this topic, you should be able to:
- Describe Lustre's usefulness in high-performance computing environments
- Explain the MDS and OSS server functions in the Lustre system
- List Lustre's components
- State two benefits of striping files across multiple OSTs
- Demonstrate controlling the striping properites of Lustre files and directories
Prerequisites
The Parallel I/O roadmap assumes that the reader has basic knowledge of Linux shell commands, parallel programming, and MPI. Coverage of these prerequisites can be found in the Shells topic plus the roadmaps on Parallel Programming Concepts and High-Performance Computing and MPI Basics.
Programming experience in C or Fortran is also recommended. Introductory roadmaps on C and Fortran are available, though the reader will need to look elsewhere for a full tutorial on these languages.
In sequence, the current roadmap logically follows the MPI Advanced Topics roadmap, but the latter is not a prerequisite.