Recommendations
It is reasonable to start programming in MPI with nonblocking calls in standard mode. Nonblocking calls eliminate the possibility of deadlock and may reduce synchronization overhead. Standard-mode communication generally gives good performance over a range of message sizes. Therefore:
Use nonblocking/standard with exceptions:
- Use a blocking call if a wait must immediately follow
- Stay with synchronous mode if memory is extremely tight
In places where the program requires a send or receive to be immediately followed by a wait, it is more efficient to use a blocking call. Also, if memory usage is a concern, it may be advantageous to write code initially for synchronous mode, then switch to standard mode. Testing in synchronous mode will ensure that the program does not depend on the presence of sufficient system buffering space, even for small messages. (Standard and synchronous modes work identically for large messages.)
Evaluate performance, analyze code:
- If nonblocking receives are always posted early, consider ready mode
- If there is too much synchronization overhead on sends, consider switching to buffered mode
The next step is to analyze the code and evaluate its performance. If nonblocking receives are always posted early, well in advance of the corresponding sends, it might be advantageous to use ready mode. For instance, if the sender has just gotten a message from the intended receiver, this can be viewed as a notification that the receiver is ready.
If there is too much synchronization overhead on the sending task and memory is ample, buffered mode may be more efficient, particularly for large messages.