Data Transfer Tools
Ben Trumbore, Adam Brazier, Linda Woodard (original author)
Cornell Center for Advanced Computing
Revisions: 11/2020, 2/2015, 7/2013 (original)
Once you understand your requirements and priorities for a given data transfer, you can choose the tool that best meets your needs. This section provides information on some of the best known and most useful transfer clients, but other, less popular options do exist.
Objectives
After you complete this segment, you should be able to:
- Distinguish between FTP and SFTP
- List the advantages of the
lftpclient over olderftpclients - Demonstrate using SCP on Stampede2 or Frontera
- Explain the differences between FTP and rsync utilities
- Describe the advantages provided by Rclone
- List the benefits of using the Globus data transfer mechanism
- Install data transfer utilities on Windows and Linux computers
- Name a potential pitfall of installing the Cygwin utility package to get rsync
- List common GUI data transfer utilities
- Name an extra step that users connecting to TACC systems should take to transfer data
Prerequisites
- Have some familiarity with the file systems and access requirements for high performance computing environments. Example environments would be those of Stampede2 and Frontera, described in the Stampede2 Environment and the Getting Started on Frontera topics. In particular, see the sections on "Storing and Moving Data" for Stampede2 and Frontera, respectively.