Filesystems
Frontera's shared storage is based on Lustre, a parallel distributed filesystem designed for high performance as well as high capacity. The Parallel I/O topic of the Cornell Virtual Workshop describes Lustre in more detail.
Frontera currently has three Lustre filesystems mounted: $HOME, $WORK, and $SCRATCH. All of these are accessible from any login or compute node, though their intended purposes and associated restrictions differ. The table below summarizes the defining features and limitations for each, including some details about a fourth filesystem called $FLASH.
Remember that the Good Conduct guidelines state that you should not stress the shared filesystems.
Filesystem Details
| Filesystem | $HOME |
$WORK |
$SCRATCH |
$FLASH1 |
|---|---|---|---|---|
| Intended Use | Storing source code and important files | Storing large files, packages, and software | Storing large files, I/O work | Supporting applications with very high bandwidth or IOPS requirements |
| Command-line Shortcuts to Access | cdh, cd ~, or cd |
cdw |
cds |
Contact TACC through ticket |
| Quota | 25GB | 1TB2 | No restrictions | 1.2PB |
| Maximum Files Allowed | 200,000 | 3,000,000 across all TACC systems | No restrictions | N/A |
| Lustre Defaults | 1 stripe, 1MB stripe size | 1 stripe, 1MB stripe size | 1 stripe, 1MB stripe size | N/A |
| Backed Up | Yes | No | No | N/A |
| Purged | No | No | Yes, all files not accessed3 in 10 days | N/A |
| Shared Between TACC Systems | No | Yes | No | No |
1 This filesystem is available upon request for users with data-driven applications.
2 This quota is on the total size for all files in the $WORK mount on TACC systems, since the filesystem is shared. For more information, see the $STOCKYARD section below.
3 Access time is based on a modification or read of the file, or execution of the file on a compute node. For more information, see the Frontera User Guide: File Systems.
$STOCKYARD
TACC has set up a shared filesystem called $STOCKYARD for their various HPC systems, which contains a $WORK directory for each system the user has an allocation on. This enables you to access certain files from any TACC system without the hassle of moving them around. For example, here are the contents of one user's $STOCKYARD directory, which can be reached from Frontera with the shortcut command cdy (or cdg):
$ cdy
$ ls $STOCKYARD
frontera stampede3
Based on this output, we can see that the user currently has an allocation on Frontera and Stampede3. On each system $WORK evaluates to the specific WORK directory of that system.
If you were to echo the $STOCKYARD and $WORK variables, you would see something like:
$ echo $STOCKYARD
/work2/12345/<username>
$ echo $WORK
/work2/12345/<username>/frontera
So $STOCKYARD/frontera is equivalent to the $WORK directory while on the Frontera system. Note that 12345 will be replaced with a different number, and <username> will be your TACC username. For more information on the shared filesystems, see the Frontera User Guide: Navigating the Shared File Systems.