Nodes needed in the cluster

In this recipe, we will look at the number of nodes needed in the cluster based upon the storage requirements.

From the initial Disk space calculations recipe, we estimated that we need about 2 PB of storage for our cluster. In this recipe, we will estimate the number of nodes required for running a stable Hadoop cluster.

Getting ready

To step through the recipe, the user needs to have understood the Hadoop cluster daemons and their roles. It is recommended to have a cluster running with healthy HDFS and at least two Datanodes.

How to do it...

  1. Connect to the master1.cyrus.com master node in the cluster and switch to the user hadoop.
  2. Execute the command as shown here to see the Datanodes available and the disk space on each ...

Get Hadoop 2.x Administration Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial