Using Hadoop Vaidya to identify performance problems

Hadoop Vaidya is an open source, rule-based performance diagnostic framework for Apache Hadoop. Each rule can identify a specific performance problem. For example, Hadoop cluster administrators can use Vaidya to identify slow progressing jobs that are wasting cluster resources. Hadoop clients can use Vaidya to identify configuration mistakes for their submitted jobs.

Hadoop Vaidya is extensible; users can analyze Hadoop job with their own rules. In this recipe, we will outline steps to configure Hadoop Vaidya for Hadoop cluster performance diagnosis.

Getting ready

Before getting started, we assume that the Hadoop cluster has been properly configured and all the daemons are running without any ...

Get Hadoop Operations and Cluster Management Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.