O'Reilly logo

Hadoop Operations and Cluster Management Cookbook by Shumin Guo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Benchmarking and profiling a Hadoop cluster

Benchmarking of a Hadoop cluster is the first step to tune the performance of a Hadoop cluster. We can also use Hadoop benchmarks to identify configuration problems and use it as reference for performance tuning. For example, by comparing the local benchmark with clusters with similar configurations, we can have a general understanding of the cluster performance.

Typically, we benchmark a Hadoop cluster after the cluster is newly configured and before putting it to service to accept jobs. This is because, when clients can submit jobs, the benchmarks can be perplexed by the client's jobs to show the real performance of a Hadoop cluster, and also the benchmark jobs can cause inconveniences for the clients. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required