Hadoop Security

Chapter 3. System Architecture

In Chapter 2, we took a look at how the security landscape changes when going from individual isolated systems to a fully distributed network of systems. It becomes immediately apparent just how daunting a task it is to secure hundreds if not thousands of servers in a single Hadoop cluster. In this chapter, we dive into the details of taking on this challenge by breaking the cluster down into several components that can independently be secured as part of an overall security strategy. At a high level, the Hadoop cluster can be divided into two major areas: the network and the hosts. But before we do this, let’s explore the operating environment in which the Hadoop cluster resides.

Operating Environment

In the early days of Hadoop, a cluster likely meant a hodgepodge of repurposed machines used to try out the new technology. You might even have used old desktop-class machines and a couple of extra access switches to wire them up. Things have changed dramatically over the years. The days of stacking a few machines in the corner of a room has been replaced by the notion that Hadoop clusters are first-class citizens in real enterprises. Where Hadoop clusters physically and logically fit into the enterprise is called the operating environment.

Numerous factors that contribute to the choice of operating environment for Hadoop are out of scope of this book. We will focus on the typical operating environments in use today. As a result of rapid advances ...