Cornell Virtual Workshop > MPI Collective Communications > Data Movement

Gather and Scatter

If an array is scattered across all processes in the group and one wants to collect each piece of the array into a specified array on a single process, the call to use is GATHER. On the other hand, if one wants to distribute the data into n segments, where the i-th segment is sent to the ith process in the group which has n processes, use SCATTER. Think of it as the inverse to GATHER.

We will first consider the basic form of these MPI gather/scatter operations, in which the number of data items collected from or sent to processes is the same for all processes, and the data items are arranged contiguously in order of process rank.

Gather and Scatter Syntax

int MPI_Gather(void *sbuf, int scount, \
     MPI_Datatype stype, void *rbuf, int rcount, \
     MPI_Datatype rtype, int root, MPI_Comm comm )

int MPI_Scatter(void *sbuf, int scount, \
     MPI_Datatype stype, void *rbuf, int rcount, \
     MPI_Datatype rtype, int root, MPI_Comm comm)

MPI_GATHER(sbuf, scount, stype, rbuf, rcount, rtype,
     root, comm, ierr)

MPI_SCATTER(sbuf, scount, stype, rbuf, rcount, rtype,
     root, comm, ierr)

Parameters for Gather routines:

sbuf: is the starting address of the send buffer
scount: is the number of elements to be sent
stype: is the data type of send buffer elements
rbuf: is the starting address of the receive buffer
rcount: is the number of elements to be received by a single process
rtype: is the data type of the receive buffer elements
root: is the rank of the receiving process
comm: is the communicator

Info: Only the root process uses rbuf, rcount, or rtype.

Parameters for Scatter routines:

sbuf: is the starting address of the send buffer
scount: is the number of elements to be sent to each process
stype: is the data type of send buffer elements
rbuf: is the address of the receive buffer
rcount: is the number of elements in the receive buffer
rtype: is the data type of the receive buffer elements
root: is the rank of the sending process
comm: is the communicator

Info: Only the root process uses sbuf, scount, stype.

Gather and Scatter Examples

In the gather operation, each process (root process included) sends scount elements of type stype of sbuf to the root process. The root process receives the messages and stores them in rank order in the rbuf. For scatter, the reverse holds. The root process sends a buffer of N chunks of data (N = number of processes in the group) so that process 1 gets the first element, process 2 gets the second element, etc.

Here we give two Fortran program fragments further showing the use of MPI_GATHER and MPI_SCATTER.

Fortran MPI_GATHER Example

In MPI_Gather, processes copy their data to the receive buffer on the root process. — In MPI_Gather, processes copy their data (25 elements) to the receive buffer on the root process.

⋮
    real a(25), rbuf(MAX)
    ⋮
    call mpi_gather(a, 25, MPI_REAL, rbuf, 25, MPI_REAL, root, comm, ierr)
    ⋮

Fortran MPI_SCATTER Example

In MPI_Scatter, each process copies a designated portion of the data from the send buffer on the root process to their respective receive buffer. — In MPI_Scatter, each process copies a designated portion (25 elements) of the data from the send buffer on the root process to their respective receive buffer.

⋮
    real sbuf(MAX), rbuf(25)
    ⋮
    call mpi_scatter(sbuf, 25, MPI_REAL, rbuf, 25, MPI_REAL, root, comm, ierr)
    ⋮

Back