Big Data, Data Mining, and Machine Learning: Value Creation for Business Leaders and Practitioners

CHAPTER 2 Distributed Systems

If you look back before electronic computers, the term “computer” meant a person who performed numerical calculations. Because roomfuls of these computers were in use for most of human history, arguably distributed (i.e., multiperson) computing dates back much farther than conventionally asserted.

Modern distributed computing arguably derives from the 1990s efforts to build “Beowulf” clusters of computers and from early ad hoc computing.¹ Beowulf clusters of computers were standard server or even desktop computers that were networked. The software on the computers then communicated closely with each other over the network to split the work to be done across the computers. Early ad hoc computing efforts, including distributed.net and SETI@Home, used idle home computers to search for cryptography keys and alien radio signals in radio astronomy data. Neither of these projects had the computing power in-house to solve the problem, nor sufficient budget to do it. However, the problems were simple enough that a large number of computers could make considerable progress with minimal coordination between them. Home computers running the distributed.net or SETI@Home software contacted the main server to get a chunk of work to be done (keys to check or radio signal data to examine).

Classical supercomputers were very large, very expensive machines that contained specialized fast hardware and processors. These supercomputers had contained in their specialized ...