Architecting Modern Data Platforms

Chapter 15. Solutions for Private Clouds

This most important thing you need to know about private clouds is that no two private clouds are the same.

If you build a private cloud in an enterprise IT context, you need to choose a combination of the following:

• A provisioning software framework, such as OpenStack, OpenShift, or Cloud Foundry

• A hypervisor, such as VMware ESX, Xen, or KVM

• A block storage solution (depending on your environment, local disks or possibly a scalable SAN infrastructure from one of the well-known enterprise storage manufacturers)

• Optionally, an object storage implementation

• Optionally, a software-defined networking (SDN) solution

Because the landscape of private clouds is so diverse, the number of possible implementations is very large, and most of the components just discussed are typically delivered by third-party vendors.

Designing your cloud to be able to run Hadoop makes everything a little bit more interesting. Your storage system should support very high throughput rates, your network should be very fast, and your provisioning logic should be smart enough to distribute all cluster roles correctly on the right hosts.

This chapter provides a rundown of the key success factors of Hadoop on private clouds, rather than offering extensive coverage of each available technology asset. For this, we assume the vantage point of the cloud architect. If you are, instead, a user of private clouds—for example, a developer of big data applications—this ...