Using Hadoop Vaidya to identify performance problems
Hadoop Vaidya is an open source, rule-based performance diagnostic framework for Apache Hadoop. Each rule can identify a specific performance problem. For example, Hadoop cluster administrators can use Vaidya to identify slow progressing jobs that are wasting cluster resources. Hadoop clients can use Vaidya to identify configuration mistakes for their submitted jobs.
Hadoop Vaidya is extensible; users can analyze Hadoop job with their own rules. In this recipe, we will outline steps to configure Hadoop Vaidya for Hadoop cluster performance diagnosis.
Before getting started, we assume that the Hadoop cluster has been properly configured and all the daemons are running without any ...