Now that you have seen how to access your files on different TACC systems, you may wish to move your source code and data from your local machine or another remote machine onto Frontera. Regardless of the size or quantity of files you must transfer, it is important to remember to limit file transfers in accordance with the Good Conduct guidelines. If you are only interested in sharing with a collaborator, see Sharing Project Files on TACC Systems.

Software Tools for Transfers

• scp, sftp, and rsync
• These commands are covered in the Data Transfer roadmap of the Cornell Virtual Workshop
• The usage of the scp command is also covered in the An Introduction to Linux roadmap and the Frontera User Guide
• Basic scp example for copying a file from a shell on your local machine to your Frontera work directory:
  Or equivalently,
  Because '$WORK' is in single quotes, it is evaluated on the remote system, Frontera.
• WinSCP for Windows users
• Globus (CLI or Globus Connect)
• See the TACC-specific information for Globus
• For a tutorial, see the Globus roadmap; there is also an Advanced Globus roadmap

Large File Sizes or File Counts

The best practice for these situations is to use compression on the file or files to reduce the overall size of the data that needs to be transferred, and uncompress at the destination. When you are transferring a directory containing many files (even small ones), they should be collected into an archive or tar file as part of this process. If you are unfamiliar with the concepts, an elementary explanation of the tar and gzip commands is present in the Linux roadmap.

Gathering and compressing files on a Windows system

Gathering files is easy in Windows with either the built-in compressed folder function, or a third-party function from the command line. Right-click on a folder and select the Send To > Compressed (zipped) folder option, which will compress the folder using the zip algorithm. If you have Cygwin or Windows Subsystem for Linux (WSL) installed, you can also do the following from the command line:

This command will create a compressed file called nmr.zip that is ready to upload. The nmr.zip file can then be decompressed on Frontera using the unzip command.

Striping Large Files

On Frontera, remember to set an appropriate stripe count on the receiving directory for large files using the Lustre command lfs setstripe. At least one stripe per 100GB in a file is recommended. The stripe count for a particular file or directory can be verified with lfs getstripe. For more information, see the Frontera User Guide: Striping Large Files.