Effective Multi-Tenant Distributed Systems

Chapter 3. CPU Performance Considerations

Introduction

Historically, large-scale distributed systems were designed to perform massive amounts of numerical computation, for example in scientific simulations run on high-performance computing (HPC) platforms. In most cases, the work done on such systems was extremely compute intensive, so the CPU was often the primary bottleneck.

Today, distributed systems tend to run applications for which the large scale is driven by the size of the input data rather than the amount of computation needed—examples include both special-purpose distributed systems (such as those powering web search among billions of documents) and general-purpose systems such as Hadoop. (However, even in those general systems, there are still some cases such as iterative algorithms for machine learning where making efficient use of the CPU is critical.)

As a result, the CPU is often not the primary bottleneck limiting a distributed system; nevertheless, it is important to be aware of the impacts of CPU on overall speed and throughput.

At a high level, the effect of CPU performance on distributed systems is driven by three primary factors:

• The efficiency of the program that’s running, at the level of the code as well as how the work is broken into pieces and distributed across nodes.

• Low-level kernel scheduling and prioritization of the computational work done by the CPU, when the CPU is not waiting for data.

• The amount of time the CPU spends waiting for data from ...