Single-threaded messaging

Call MPI from single-threaded tasks (not hybrid; does not require MPI to have thread support) Call MPI from just the main thread—either in a serial region, or after selecting it in a parallel region Call MPI from multiple threads—but synchronized, so that only one thread calls MPI at a time

Multithreaded messaging

Call MPI from multiple threads anywhere in a parallel region; MPI sends and receives messages in parallel Requires an implementation of MPI that is fully thread-safe

In the figure, each node in a pair of nodes is assumed to be occupied by a single MPI task, identified by its rank. Two different situations are shown. At the top, a single thread on one of the MPI ranks exchanges messages with a single thread on the other rank—perhaps even in circumstances where multiple threads are running. At the bottom, some or all the threads on one MPI rank are able to exchange messages simultaneously with partner threads on another MPI rank. The messages in the second situation will likely need to be identified by thread ID as well as rank.

You'll want to keep the above picture in mind as you make your way through this topic. Remember, only in the bottom part of the figure are the MPI messages being sent in parallel among multiple threads on two (or more) MPI ranks.