Slurm offers a dependency mechanism that allows conditional execution of a job based upon some characteristic pertaining to the execution of another job. If a job is specified as being dependent on another, the dependent job is blocked in the queue until the dependency condition is achieved. The job executes or is cancelled based on evaluating this condition. Through dependencies, control flow between jobs can be established.

Dependency Conditions

after

Indicates that the job may begin after the dependent job(s) have started executing. Using this, jobs may be ordered with respect to their starting times.

afterany

Indicates that the job may begin after the dependent job(s) have terminated, regardless of how they were terminated (e.g., success, failure, cancellation).

afterok

Indicates that the job may begin after the dependent job(s) have run to completion successfully with an exit code of 0.

afternotok

Indicates that a job may begin after the dependent jobs(s) have terminated, and at least one of the jobs has failed (return code > 0) or terminated abnormally.

expand

Indicates that the resources allocated to this job are to be added to those of the dependent job. Additional steps are necessary in order to expand the dependent job to use the new resources. Only one job may be named in an 'expand' dependency; a job cannot expand the resources of multiple jobs. (This option is disabled by default and is not enabled on Stampede2 or Frontera.)

singleton

Assures that only one job with the same name, and from the same user, executes while this job is running. If any other jobs with the same name and user are already executing, the current job will block until those jobs terminate for any reason.

For those dependencies that accept multiple job IDs as an argument (all but 'expand' and 'singleton' in the above list), the pertinent job IDs may be provided as a colon-separated list. For example, --dependency=afterok:100:101:102:103:104:105 indicates that the job shall start only after jobs 100-105 successfully complete.