Funneled MPI via Main Thread
Below is a different example of a hybrid code that requires MPI_THREAD_FUNNELED
support or higher. At this level of MPI thread support, only the main thread may make MPI calls (i.e., MPI calls must be "funneled" through the main thread).
Some notes on this program:
- In OpenMP, the main thread can be selected by the
omp master
construct - This will work, provided that the current thread team includes the main thread of the whole program
-
The
omp master
construct does not create implicit barriers for the other threads, so it may also be necessary to useomp barrier
-
As illustrated in the example, barriers may be required in two places:
- Before the MPI call, in case the MPI call needs to wait on the results of other threads
- After the MPI call, in case other threads immediately need the results of the MPI call
- The example shows the main thread executing a single MPI call, while the other threads are idle
Note, the main thread can also be selected by testing the result of a call to MPI_Is_thread_main()
. This test is appropriate if MPI_Init_thread()
was called from an arbitrary thread, say the one chosen by omp single
in a multithreaded region. The thread selected by this test is the main thread from the point of view of MPI, even if it is not the main thread from the point of view of OpenMP. Similarly, the MPI_Is_thread_main()
test is appropriate inside two or more nested omp parallel
constructs, because in that case there is not a single, unique OpenMP main thread. Calling MPI_Is_thread_main()
is always thread-safe.