Configure Oozie and workflows

In this recipe, we will configure the Oozie workflow engine and look at some examples of scheduling jobs using the Oozie workflow.

Oozie is a scheduler to manage Hadoop jobs, with the ability to make decisions on the conditions or states of previous jobs or the presence of certain files.

Getting ready

Make sure that you have completed the recipe of Hadoop cluster configuration with the edge node configured. HDFS and YARN must be configured and healthy before starting this recipe.

How to do it...

  1. Connect to the edge node edge1.cyrus.com and switch to the hadoop user.
  2. Download the Oozie source package, untar it, and build it:
    $ tar –xzvfoozie-4.1.0.tar.gz
$ cd oozie-4.1.0
  3. Edit the file pom.xml to make it suitable for the Java ...

Get Hadoop 2.x Administration Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial