Cornell Virtual Workshop > Parallel Programming Concepts and High Performance Computing > Introduction to Parallel Computation

Working Independently

When you have a group of people working together, it is often more efficient to use a different process than you would for a single person. If you gave each worker their own workspace and half the card deck, they could work at full speed with no resource contention. A workflow design where parallel workers have independent workspaces is called distributed memory parallelism in parallel computing.

Multiple independent workers

While the individual sorting strategy is the same for all workers, giving workers independent workspaces changes the workflow. In contrast with the previous workflows, each worker:

has independent inputs
works in their own workspace
produces independent output

This design solves the resource contention, but it also creates a new kind of work! You want one final result, but you have one output per worker. As the deck owner (manager), you are responsible for combining these outputs into a coherent final result. The amount of work required to coordinate multiple workers can increase rapidly with the number of workers.

The unsorted deck is randomly divided between two workers. Each worker produces sorted output but the manager must merge these sorted outputs into a final result. — When cards are randomly divided among workers to sort, each worker produces one sorted stack of cards for each suit. The manager (you!) must collect and merge these outputs. Depending on how work is distributed and the number of workers, the effort to coordinate their output can be significant.

Back