Summary
In Approach #1, we used MPI_Comm_split to split one communicator into multiple, non-overlapping communicators.
- This approach is compact and is suitable for regular decompositions.
In Approach #2, we broke the communicator into (sub)groups and formed these into new communicators to suit our needs.
- We did this using MPI_Comm_group, MPI_Group_incl, and MPI_Comm_create.
- MPI_Comm_create_group provides a noncollective alternative to MPI_Comm_create, suitable for large parent communicators.
- This approach is quite flexible and is more generally applicable.
- Many other group functions are available: union, intersection, difference, include, exclude, range-include, range-exclude.
Approach #3 builds largely on Approach #2 as well as point-to-point communication.
- Create "normal" intra-communicators.
- Next, link these two groups using point-to-point communication by picking a leader process in each intra-communicator. This is done with MPI_Intercomm_create.
- Inter-communicators can be merged back to an intra-communicator, if desired, using MPI_Intercomm_merge.
This approach is useful when two communicators may have different purposes over time and when there are two tiers of logic happening in a communicator, where one tier has different functionality on different processes.