Here is a short example of persistent communication. We'll walk through this example in the paragraphs below.

Step 1: Create requests

MPI provides special _init commands to create the send and receive objects without actually sending any messages. The init methods should look very similar to the Send and Recv commands discussed earlier, so we won't go over the parameters to these functions. All persistent communication objects are nonblocking, but all 4 communication modes are still available and named as expected (MPI_Ssend_init, MPI_Bsend_init, etc). All of these calls populate an MPI_Request object for use later.

Step 2: Send and receive messages

Persistent communication messages are sent and received using the MPI_Start command, which simply takes an MPI_Request handle (a reference to the request object) and performs the prescribed operation. MPI_Start returns immediately, so in the example above, the receive is nonblocking but the send emulates a blocking communication call, because the Wait was called directly after the MPI_Start. The MPI_Wait method should look familiar: once MPI_Start is called, the request can be treated like any other nonblocking communication. The only caveat is that a single request object cannot track multiple communications, so a Wait or Test must be used before the object is re-used. On the other hand, if multiple request objects have been defined through calls to _init routines, an array of these requests can be kicked off at once with a call to MPI_Startall.

Step 3: Clean up objects

The programmer is obligated to clean up persistent objects, as they are not deallocated by Wait or Test calls. The MPI_Request_free method explicitly deallocates the request objects once you are done with them.