O'Reilly logo

Hadoop Operations and Cluster Management Cookbook by Shumin Guo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Configuring a Hadoop cluster with the new AMI

Starting a Hadoop cluster with the new AMI is simple and straightforward. This recipe will list steps to start up a Hadoop cluster with the new AMI.

Getting ready

Before getting started, we assume that you have registered with AWS and have successfully created a new AMI with Hadoop properly configured.

How to do it...

Use the following steps to configure a Hadoop cluster with EC2:

  1. Run a number of instances either from the command line or from the web interface.
  2. After the instances are all in running state, run the following command to get the internal hostname of these instances:
    ec2-describe-instances | grep running | egrep -o 'ip.*?internal' | sed -e 's/.ec2.internal//g' > nodes.txt
    

    The nodes.txt file ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required