Parallel Computing for Data Science

Chapter 8

The Message Passing

Paradigm

The scatter-gather paradigm we’ve seen in earlier examples with snow

works well for many problems, but it can be confining. This chapter will

present a more general approach.

Instead of a situation in which the workers communicate only with the

manager, as in for instance in snow, think now of allowing the workers

to send messages to each other as well. This general case is known as the

message passing paradigm, the subject of this chapter.

8.1 Message Passing Overview

A message-passing package will have some kind of send() and receive()

functions for its basic operations, along with variants such as broadcasting

messages ...