Suppose you could enlist two people (workers) to help you. These two workers can take your place at the sorting table, picking up cards and inserting them into the appropriate stack. The two workers are a direct replacement for the single worker in the first example:

• they work from one pile of unsorted cards.
• all workers have a complete view of the current progress and activity.
• they produce one set of sorted cards.
• they use the same strategy as the single worker.

The critical difference is that these workers are both working at the same time (in parallel).

Multiple people sharing a sorting table

In this setup, think of the two workers as sharing an office cubicle that previously held one worker. The cubicle has one in-box, one out-box, and one work surface. The workers share access to the tabletop, so they see the same unsorted cards and use the same working space to sort the cards. We will use the term workspace to describe the resources allocated to support a worker's activities. A workspace includes room to work and some way to send and receive information.

A workflow where parallel workers share a workspace is called shared memory parallelism in parallel computing.

While adding parallel workers to a workspace can be an easy way to get the work done more quickly, this approach has limitations. When a worker picks up a pile to insert a card, that pile is unavailable to other workers until the insert is complete. As long as the workers are processing cards of different suits, all the workers can process cards at full speed. On the other hand, if both workers need to insert cards from the same suit, one worker will pick up the deck first and the other will have to wait.

Resize Image

In parallel computing, when one worker waits for another to finish using a resource (like a pile of cards or writing to memory), it is called resource contention. In our example, resource contention limits the number of workers that can share a sorting table. With two workers, resource contention might be infrequent. With more than four workers (more than one worker per suit), at least one worker will always be waiting! Even if you had a hypothetical card deck with many more suits, the table top capacity is finite and the workers will eventually run out of space to work.

In workflows based on shared memory parallelism, the amount of resource contention determines the optimal number of workers, and the overall size of the workspace limits the size of the card deck (amount of data) the workflow can process.