It is recommended that you transfer large files sequentially.

When transferring a large file, Globus will automatically distribute the work over multiple communication streams if possible. This will result in the transfer completing more quickly than it would have using a single stream. If a user tries to achieve faster transmission of multiple files by starting multiple transfers at once, the transmission of each file will probably be slower, and could result in a similar total time to completion. Additionally, such multiple simultaneous transfers may use an inordinate percentage of the available bandwidth, adversely affecting other users.

Use tar to package many small files into a single large file before transferring.

It is important that users understand the physical characteristics of the TACC resources that are being used in Globus data transfers. Each resource may dictate its own set of best practices to provide the most efficient transfers. For example, files transferred to the TACC resource "Ranch", will be written to tape storage, which is a slow process that is much more efficient when dealing with fewer, larger files. A specific recommendation for Ranch is to use tar to package many small files into a single large file before transferring, and this advice is also relevant on other computer systems.

 
©   Cornell University  |  Center for Advanced Computing  |  Copyright Statement  |  Inclusivity Statement