• A great deal of hidden communication takes place with collective communication
• Performance depends greatly on the particular implementation of MPI
• Because there may be forced synchronization, it may not always be best to use collective communication

MPI implementors have already considered many situations like those on the following pages. Even though this is already handled behind the scenes, it is still useful to understand what might be happening when designing your own algorithms that use MPI.