O'Reilly logo

Hadoop 2.x Administration Cookbook by Gurmukh Singh

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Namenode roll edits – online mode

In this recipe, we will take a look at how to roll edits and keep the size of the metadata to a minimum. Over a period of time, the number of edits files grow and also the Namenode keeps old versions of fsimage file. In a large busy cluster, the edits files could be large, with each of about 1 GB.

This will utilize disk space on Namenode and can cause disk issues in the longer run of the cluster. Also, if the Secondary Namenode is not configured or is not working correctly, these edits files will be in large numbers, with each file of approximately 1 million transactions. Due to this, the Namenode start time will increase and Namenode might not even start if the memory is not sufficient to do the checkpoint operation. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required