Syntax:

• Input: rqd, or "required" (integer)
  • Indicates the desired level of thread support
• Output: pvd, or "provided" (integer, passed by reference in C)
  • Indicates the level of thread support available to the program
  • If thread level rqd is supported, the call generally returns pvd = rqd
  • Otherwise, pvd returns the highest provided level of support
  • In C++, use the C bindings
• Note, there is no guarantee that pvd will be greater than or equal to rqd

The following C program should show the highest level of thread support for a given MPI implementation. If you compile it with mpicc and run it on Frontera or Stampede3, you should find that their MPI libraries provide the broadest level of support, MPI_THREAD_MULTIPLE.

If you substitute a lower level than MPI_THREAD_MULTIPLE in the above call to MPI_Init_thread(), and then recompile and re-run the code, you should find that *thread_support is set to match the level you required on Frontera and Stampede3.

While most of today's MPI implementations are fully thread safe, it can still be a good idea for a hybrid program to do a safety check like the one above after calling MPI_Init_thread(). A particular risk occurs when the MPI implementation was built from source. If the step that adds thread support to MPI was omitted when the MPI library was built, then the resulting MPI implementation will have only MPI_THREAD_SINGLE capability. Therefore, a hybrid program might fail. But at least it can exit gracefully, or provide a fallback, if it detects *thread_support==MPI_THREAD_SINGLE upon return from MPI_Init_thread().

What about a multithreaded code that only calls MPI from serial regions? This type of fork-join parallelism is actually typical of many OpenMP codes. Requiring MPI_THREAD_FUNNELED should be sufficient in that case. Examples of this hybrid code type can be found elsewhere in the Virtual Workshop, in the Getting Started on Frontera and Introduction to Advanced Cluster Architectures roadmaps.

For a more complex hybrid code that is designed to send MPI messages between multiple individual threads, the best place to start is to require MPI_THREAD_MULTIPLE. However, if the multithreaded MPI calls have all been serialized by putting them in OpenMP critical sections (e.g.), there can be a benefit to requiring MPI_THREAD_SERIALIZED. This notifies MPI of the lower level of support that is really required. MPI may then be able to skip some of the thread initializations and internal locks that would otherwise be needed, which may improve its messaging performance. Only experimentation can say for sure which value of rqd works best in such cases.

Each of the levels of MPI thread support will be described in greater detail on the pages to follow, with examples.