Configuring a Hadoop cluster with the new AMI
Starting a Hadoop cluster with the new AMI is simple and straightforward. This recipe will list steps to start up a Hadoop cluster with the new AMI.
Before getting started, we assume that you have registered with AWS and have successfully created a new AMI with Hadoop properly configured.
How to do it...
Use the following steps to configure a Hadoop cluster with EC2:
- Run a number of instances either from the command line or from the web interface.
- After the instances are all in running state, run the following command to get the internal hostname of these instances:
ec2-describe-instances | grep running | egrep -o 'ip.*?internal' | sed -e 's/.ec2.internal//g' > nodes.txt