Nodes needed in the cluster

In this recipe, we will look at the number of nodes needed in the cluster based upon the storage requirements.

From the initial Disk space calculations recipe, we estimated that we need about 2 PB of storage for our cluster. In this recipe, we will estimate the number of nodes required for running a stable Hadoop cluster.

Getting ready

To step through the recipe, the user needs to have understood the Hadoop cluster daemons and their roles. It is recommended to have a cluster running with healthy HDFS and at least two Datanodes.

How to do it...

  1. Connect to the master node in the cluster and switch to the user hadoop.
  2. Execute the command as shown here to see the Datanodes available and the disk space on each ...

Get Hadoop 2.x Administration Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.