Sync Files and Verify Transfer

As mentioned in the tool descriptions, a few tools provide unique options that can greatly speed up or slow down a data transfer. Think carefully whether you can benefit from such savings or whether you really need the additional time cost.

Choosing to Sync Instead of Copy

If you are updating the contents of a directory, consider using rsync (or, if an ftp server is available, lftp with mirror), which only transfers the changed parts of the directory. This can be a tremendous time saver.

Verifying Large Transmitted Files

The possibility of transmission errors increases with the size of the file being transferred, so you may want to verify that the source and destination files are the same after a transfer. An easy way to determine if two files are the same is to run the sha256sum command on a file before and after it is transferred and compare the checksums calculated. Even a one bit difference will result in different checksums.

sha256sum fileToBeTransferred

As we've seen, several transfer utilities can perform such verification for you as part of the data transfer.

 
©  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Access Statement