snow stands for Simple Network Of Workstations. It is built on top of Rmpi, but you don't need to be familiar with MPI to use this library. It is a good choice for embarrassingly parallel applications that can use more than one core. It relies on a manager/worker model where one process (the manager) controls the rest of the processes which do the actual work. These workers communicate their results back to the manager process, which may do more work on the aggregated data. Note that in these examples we are not requesting as many tasks as the system supports; this is simply to make the output, which involves each task printing to file, easier to read; you should feel free to increase the number of requested tasks.

On Stampede2, TACC offers convenient environment modules that help you to create the proper setup for snow. Thus, you no longer need to install the snow package on Stampede2 before you can start to use it. Instead, you just do , which gives you access to a fairly recent version of R plus a number of useful packages, including snow. (If you are running your R code on a different system, or if you are using a different version of R on Stampede2, please refer to the Additional Packages page for instructions on how to install snow.)

In the simplest case, the cores on a single node may be sufficient for your application. The following short example illustrates how to use snow in an interactive session on Stampede2. It is taken from a presentation titled "Introduction to R" by David Walling at TACC (who kindly provided a few of the examples on this page). The goal of the code is to simulate a classic problem from probability theory: if there are n people in a room, how likely is it that 2 or more of them share the same birthday?

Below is the birthday.R code. It includes snow functions that serve to distribute the major calculations to an intra-node cluster of workers.

Here is how to run the code interactively on Stampede2, assuming you have loaded the relevant modules (if not, you should load them first as described in the Configure R on Stampede2 or Frontera page in the Running R on HPC systems roadmap):

Note that, as per Stampede2 default guidance, $SLURM_NPROCS is set to 68. If you don't wish to view the line-by-line progress of the code, you can run the script with the command .

What if you want to run workers on more than one node? That's where Rmpi comes in. Happily, the task of incorporating Rmpi is simplified for you through a wrapper application called RMPISNOW. It hides a lot of MPI's messiness. The following script will let you try RMPISNOW in batch , so you can see how to run snow across multiple nodes. Construct the two files given below, and , and then just submit the job with , and it should run SimpleSNOW.R on 2 Stampede2 nodes with a total of 23 workers plus one manager.

In the output, notice that there are 23 workers, even though we requested 24 processes. As advertised above, that is because RMPISNOW assumes that the manager process requires a full core to fulfill its role. On a single node, though, we are able to call snow's function, which lets us start as many workers as we want, up to (or even beyond) the full number of cores if desired.

You should be aware that there is a second package called snowfall which provides an even friendlier wrapper around snow. It can make developing parallel R code somewhat easier. To get access to it on Stampede2, just do "module load Rstats" as you would for snow; on other systems, you will need to install snowfall as well as the snow package.

The following batch script serves as an example of how to use snowfall on a single Stampede2 node. You can call it . Notice the use of the Unix "hereis" () shell syntax to combine the batch script and the R script into one file.

Just submit the job as normal:

Note that we request only one node: apparently, snowfall does not play well with RMPISNOW. Thus, there seems to be no easy way to run snowfall jobs that span multiple cluster nodes. You can find more information on snowfall at CRAN.