Storage Limitations
Each TACC resource provides one or more file systems into which you can transfer files, and it is important to understand the size limitations of the file systems you are using. For instance, the File systems on the Stampede3 resource provide:
- A backed-up $HOME directory for your active files, currently limited to 5 GB.
- A non-backed-up $WORK directory for large files, currently limited to 1 TB.
- A non-backed-up temporary $SCRATCH directory for transient files.
If a user tries to perform a Globus transfer on a file that is larger than the space remaining in their TACC $HOME quota, the transfer will fail repeatedly.
Users can also control the Lustre striping that is used on the TACC resources where they store files. Striping options control how a file is distributed across the multiple storage targets (OSTs) that make up a file system. If a user who has not changed the default settings transfers a huge file, the system may attempt to store it all in a OST, which in an extreme case could crash the entire file system. To avoid such problems, it is recommended that users use appropriate striping settings when transferring very large files. If the documentation alone cannot answer questions about such issues, contact the online help facilities for the resource.