Moving Hadoop to the Cloud

Chapter 8. Setting Up in Azure

In this chapter you’ll create a simple Hadoop cluster running in Azure. The cluster will consist of basic installations of HDFS and YARN, which form the foundation for running MapReduce and other analytic workloads.

This chapter assumes you are using a Unix-like operating system on your local computer, such as Linux or macOS. If you are using Windows, some of the steps will vary, particularly those for working with SSH.

Tip

If you just worked through a previous chapter on AWS or Google Cloud Platform, you’ll find that this chapter covers the same procedures, just under Azure. If you’re more interested in using your AWS or Google Cloud Platform cluster, skip ahead to Chapter 9.

Prerequisites

Before you start, you will need to have an account already established with Azure. You can use your current Microsoft account, or register for a separate account for free.

Free Trial

After signing up for Azure, you are eligible for a limited-time free trial, during which you are granted a fixed amount of credit that can be used to provision resources. The free trial allows you to explore Azure, but the limited number of permitted cores per region may be so low that you are unable to field a Hadoop cluster. If so, contact Azure Support to request a limit increase.

Once you are registered, you will be able to log in to the Azure portal, a web interface for using all of the different services under the Azure umbrella. A view of the portal is shown in ...