Task IDs

If your script performs a Globus file transfer, it should accept responsibility for ensuring that the transfer is successful and for taking appropriate measures if it is not. To do this, the script must remember the unique identifier for each transfer so that its status can be tracked and it can be canceled if need be.

To capture the Globus task ID that is output by a transfer command, use the --jq option to specify a JMESPath query for processing the output, in this case "task_id" to select the "task_id" field. You must also specify the output format as --format=UNIX in order to remove quotes from the string. This example saves the task ID into a variable.

Task completion and status

To make your script wait until a transfer completes before moving on to other operations, use the globus task wait command. The command takes a task ID as an argument and its -timeout option limits how many seconds it will wait for completion before exiting. Because Globus will continue to retry an incomplete transfer indefinitely, attempting to transfer a missing source file will never finish if you fail to include a timeout value.

The globus task show command reports current state information for a specific task, and can be used to determine if a completed task succeeded or failed. The "status" command returns a lot of text, but as with the "transfer" command you can extract just the part related to its success using the --jq and --format options. In this example, using --jq "status" limits the output to be the value of the status property, which will be ACTIVE, INACTIVE, SUCCEEDED or FAILED. This value can then be captured in a variable and used in the script logic.

This example is presented as an excerpt from a script, continuing from the task commands above:

# Wait until the transfer is complete before continuing.
globus task wait --timeout 30 $id

# Get the status value for the transfer operation and see if it succeeded.
status=$(globus task show --jq "status" --format=UNIX $id)
if [ $status != "SUCCEEDED" ]; then
    echo "Transfer failed!"
    exit
fi

# Otherwise, continue with script

Scripting a multi-file transfer

As we saw in the section on transfers, a single Globus CLI transfer command can operate on multiple independent files and folders. However, a single ID is still returned for the operation, regardless of how many files and folders are included in the transfer.

In a script, it is easy to construct a multi-file transfer without creating a separate file that lists the items to be copied. Here, the output from a series of "echo" commands is piped to the globus transfer command right in the script:

{
    echo "/share/godata/file1.txt file1.txt"
    echo "/share/godata/file2.txt file2.txt"
    echo "/share/godata/file3.txt file3.txt"
} | globus transfer --batch - $gte1 $gte2