Parallel Computing for Data Science

2.5. LATENCY AND BANDWIDTH 27

memory becomes the vehicle for communication between the various pro-

cesses.

Your program consists of a number of machine language instructions. (If

you write in an interpreted language such as R, the interpreter itself consists

of such instructions.) As the processors execute your program, they will

fetch the instructions from memory.

As noted earlier, your data—the variables in your program—is stored in

memory. The machine instructions fetch the data from memory as needed,

so that it can be processed, e.g., summed, in the processors.

Until recently, ordinary PCs sold at your local electronics store followed

the model