O'Reilly logo

Hadoop 2.x Administration Cookbook by Gurmukh Singh

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Queuing mappings in Capacity Scheduler

In this recipe, we will be configuring users who can submit jobs to the queue and can also set rule for various job submissions.

Let's look at another use case where, if user hadoop submits a job, it should go to the prod queue and if any other users submits a job, it must go to dev queue. How can we set up something like this?

Getting ready

Make sure that the user has a running cluster with HDFS and YARN configured. It's best to have gone through at least the previous recipe.

How to do it...

  1. Connect to the master1.cyrus.com Namenode and switch as user hadoop.
  2. Edit the capacity-scheduler.xml allocation file as shown next:
    <property> <name>yarn.scheduler.capacity.queue-mappings</name> <value>u:d1:dev,g:group1:default,u:hadoop:prod</value> ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required