Monitoring a Hadoop cluster with Chukwa

Chukwa is a project developed for collecting and analyzing Hadoop logs. It uses HDFS as its storage architecture and contains a number of toolkits for log analysis and cluster monitoring. In this recipe, we will guide you through steps to configure Chukwa to monitor a Hadoop cluster.

Getting ready

The latest release of Chukwa uses HBase for key-value storage. So, before getting started, we assume that Hadoop and HBase have been installed and properly configured.

Next, we can use the following steps to install Chukwa:

  1. Download the latest release from the official website http://incubator.apache.org/chukwa, for example, use the following command:
    sudo wget http://mirror.symnds.com/software/Apache/incubator/chukwa/chukwa-0.5.0/chukwa-incubating-src-0.5.0.tar.gz ...

Get Hadoop Operations and Cluster Management Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.