Lustre Striping

In Lustre, a logical object volume serves as an umbrella for multiple Object Storage Clients (OSCs), each of which connects to an OST. Because of this structure, a single file may be split across several OSTs, in a technique called striping. Striping allows parts of files to be stored on different OSTs in a RAID-0 pattern. This is done to enable fast parallel I/O, just as in a RAID-0 disk array.

  • The number of objects is called the stripe_count.
  • Objects contain "chunks" of data that can be as large as stripe_size.
In this example of Lustre striping, one file is split across two <abbr>OST</abbr>s and another file is split across three.
The LOV is backed by one or more OSCs. Each OSC connects to a dedicated OST. Striping is a strategy that stores parts of a file on different OSTs. In this example, File A is split across two OSTs and File B is split across all three OSTs.

With striping, the performance of a Lustre file system potentially scales with the number of OSTs available. Furthermore, the capacity of a Lustre file system equals the sum of the capacities of the storage targets.

These qualities result in two benefits:

  1. I/O rate to a file is the aggregate of I/O rates to objects.
  2. Max file size is not limited by the size of a single target.

For TACC systems, the total amount of Lustre storage is measured in petabytes, and the nominal aggregate data rate exceeds 150 GB/s. But this speed is split among all users of the system. Metadata access can be a bottleneck, so the MDSs need to have especially good performance (e.g., solid state disks on some systems).

 
©  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Inclusivity Statement