Running an interactive job can be an efficient way to test or debug an application or a batch script. Here is the procedure:

  1. From a shell on a login node, request an interactive job using idev or srun.
  2. Wait. Your current shell goes into a pending state until your job starts. (If the chosen queue is busy, your wait may not be short, so check first to make sure resources are available.)
  3. Once your job starts running, your shell is connected automatically via SSH to one of the nodes allocated to your job.
  4. Enter commands as desired. Compute-intensive commands may be included.
  5. When you are finished, log out from the node. Slurm automatically terminates your job.

Common things to try interactively are to execute the commands in a batch script one at a time and observe the output, or start an MPI run with ibrun and see if the MPI tasks launch as expected. Working in this way allows you to catch problems as they occur and make modifications in response.

idev

TACC's idev utility is the most convenient way to request an interactive session on one or more compute nodes. Once the scheduler allocates the resources, you are automatically ssh'd to one of the nodes, where you can run compute-intensive commands.

The default for idev is to request a thirty-minute session on a single node in the development queue. To launch such a session, simply execute:

$ idev

You'll start to see job status messages indicating that your interactive session is waiting in the queue. When your session begins, you'll get a command prompt on a compute node. (Note, the first time you launch idev, you may be prompted to choose a few defaults for future idev sessions.)

An idev request can easily be customized using Slurm-style syntax (type idev --help to get info on the options):

$ idev -p normal -N 2 -n 8 -m 150 # normal queue, 2 nodes, 8 total tasks, 150 minutes
srun

Another way to launch an interactive session is to use Slurm's srun command. The equivalent of the pair of idev commands above would look like this:

$ srun -p development -t 0:30:00 -N 1 -n 56 --pty /bin/bash -l  # same as plain idev
$ srun -p normal -N 2 -n 8 -t 2:30:00 --pty /bin/bash -l  # same conditions as above

Notice that with srun, all of Slurm's required options must be present on the command line, plus two extras:

  • --pty tells Slurm to run a pseudo terminal on the first allocated node
  • /bin/bash -l says to open a login shell there

Perhaps the biggest difference between idev and srun is that idev gives you lots more helpful feedback on how your interactive job is configured and how it is progressing through the queue. Execute idev --help to see all the features.

 
©  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Inclusivity Statement