SCP (secure copy protocol) is one of the simplest transfer protocols to use, and is suitable for smaller transfer sizes (<= 100 GB). For larger amounts of data, its slow speed will lead to long transfer times and increase the likelihood of failures. SCP uses Secure Shell (SSH) to transmit encrypted versions of your credentials and data, increasing security.

You can perform transfers using the scp command line tool (openbsd.org man page). It takes two arguments, the source and the destination. Both can be a file or folder that is either local or remote, and wildcards are allowed. Providing credentials for transfers between two remote locations can be difficult, so the command is typically used to transfer files between the local system and one remote system.

Below is an example of a transfer of a local file to the Frontera system. A remote source or destination usually includes your username on that system, encoded as remoteUserName@remoteSystemName:path. If the username is not provided, your current login name will be used and must match the login on the remote system. The remote path can be absolute (if it starts with a "/") or relative to your home directory on the remote machine. This example transfers the file test.log from the current local directory to a file named file.txt in the folder $HOME/Examples (the folder must exist first) on Frontera:

    % scp test.log my_login@frontera.tacc.utexas.edu:Examples/file.txt

    To access the system:

    1) If not using ssh-keys, please enter your TACC password at the password prompt
    2) At the TACC Token prompt, enter your 6-digit code followed by .

    (my_login@frontera.tacc.utexas.edu) TACC Token Code:
    test.log                                                      100%    0     0.0KB/s   00:00
    

The command will ask for your password on the remote system before performing the transfer. If you are connecting to the TACC system Frontera, you will also be asked to supply a TACC token that can be retrieved from a phone app. After the transfer is complete, the command prints confirmation information including file names, sizes and transfer durations.

You can avoid having to supply your password for each transfer if you set up a passwordless SSH key pair. You do this by adding your public key to the $HOME/.ssh/authorized_keys file on the remote system and supplying your private key through the -i command option. When connecting to TACC systems you will still need to enter the TACC token from the phone app to initiate the transfer.

To copy a file while keeping the same name on the remote system, just provide the path to the destination directory. Here, the directory is omitted and is taken to be $HOME:

The same address format can also be used for the source location, so if you have logged in to Frontera you can use SCP to "download" a file from your local system. However, this works only if Frontera can locate your local machine via DNS and your computer is set up to accept SSH connections, which may not be true in every case.

The download command would be:

When transferring a whole folder of data, use the -r option to specify a "recursive" copy:

If the SCP tool is not already installed on your system, you can install it yourself. There are also a number of excellent GUI clients that make it even easier to use SCP.

Exercise

In order to perform this exercise, you must first generate the test data on Frontera. Use scp to transfer each of the three files from Frontera to your local machine (hint: use scp on your local machine) and note how long each transfer takes. The largest file, biggestfile.txt, is well below the 100 GB limit recommended for using scp. Now use scp to transfer these files back to Frontera from your local computer under different names.