O'Reilly logo

Hadoop MapReduce v2 Cookbook - Second Edition by Thilina Gunarathne

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 3. Hadoop Essentials – Configurations, Unit Tests, and Other APIs

In this chapter, we will cover:

  • Optimizing Hadoop YARN and MapReduce configurations for cluster deployments
  • Shared user Hadoop clusters – using Fair and Capacity schedulers
  • Setting classpath precedence to user-provided JARs
  • Speculative execution of straggling tasks
  • Unit testing Hadoop MapReduce applications using MRUnit
  • Integration testing Hadoop MapReduce applications using MiniYarnCluster
  • Adding a new DataNode
  • Decommissioning DataNodes
  • Using multiple disks/volumes and limiting HDFS disk usage
  • Setting the HDFS block size
  • Setting the file replication factor
  • Using the HDFS Java API

Introduction

This chapter describes how to perform advanced administration steps in your Hadoop cluster, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required