O'Reilly logo

Hadoop Operations and Cluster Management Cookbook by Shumin Guo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Using Hadoop Vaidya to identify performance problems

Hadoop Vaidya is an open source, rule-based performance diagnostic framework for Apache Hadoop. Each rule can identify a specific performance problem. For example, Hadoop cluster administrators can use Vaidya to identify slow progressing jobs that are wasting cluster resources. Hadoop clients can use Vaidya to identify configuration mistakes for their submitted jobs.

Hadoop Vaidya is extensible; users can analyze Hadoop job with their own rules. In this recipe, we will outline steps to configure Hadoop Vaidya for Hadoop cluster performance diagnosis.

Getting ready

Before getting started, we assume that the Hadoop cluster has been properly configured and all the daemons are running without any ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required