Slurm has quite a few options that let you assign different kinds of attributes to your jobs. The table below covers some of the options that might help you keep track of the progress of your jobs. These flags allow you to give a name to a job and its output streams, and to set up email notifications.

Options to track the progress of jobs.
Option Flag Value Example
Job name -J any string -J SimpleJob
Stdout -o (path and) filename -o $HOME/project1/%j.out
Stderr -e (path and) filename -e $HOME/project1/%j.err
Email address --mail-user= email@domain --mail-user=genius@gmail.com
Email event type --mail-type= {BEGIN, END, FAIL, ALL, etc.} --mail-type=ALL (email notification at start and end of job, or failure)

Full information on these options (and many more) is of course available by viewing the sbatch man page. All flags can be used either on the command line or in a batch script.

Notice that the job attributes can include special variables whose value is not known until run time. For example, the variable %j above is ultimately replaced by the job number. Linux environment-variables won't work for this purpose, as the usual variable substitution will not take place. Furthermore, Slurm environment variables like $SLURM_JOB_ID are defined only when the job is running, not when it is submitted.

Slurm "job arrays" (defined through the -a option) are not allowed on Frontera. Given that all queues put a limit on the number of jobs that can be submitted at once, you will need a mechanism beyond Slurm if your goal is to run many small jobs. That is why TACC offers the utilities mentioned earlier: launcher, launcher_gpu, pylauncher, and gnuparallel. Also, the example batch script on the next page shows a different way of launching multiple runs from a single script.

 
©  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Inclusivity Statement