O'Reilly logo

Hadoop 2.x Administration Cookbook by Gurmukh Singh

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 2. Maintaining Hadoop Cluster HDFS

In this chapter, we will cover the following recipes:

  • Configuring HDFS block size
  • Setting up Namenode metadata location
  • Loading data into HDFS
  • Configuring HDFS replication
  • HDFS balancer
  • Quota configuration
  • HDFS health and FSCK
  • Configuring rack awareness
  • Recycle or trash bin configuration
  • Distcp usage
  • Controlling block report storm
  • Configuring Datanode heartbeat

Introduction

In this chapter, we will take a look at the storage layer, which is HDFS, and how it can be configured for storing data. It is important to ensure the good health of this distributed filesystem, and make sure that the data it contains is available, even in the case of failures. In this chapter, we will take a look at the replication, quota setup, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required