Preface
Hadoop is fast becoming the de facto big data platform across all industries. An entire ecosystem of tools, products, and services targeting every functionality and requirement have sprung up around Hadoop. Apache Oozie occupies an important space in this ever-expanding ecosystem. Since Hadoop’s early days at Yahoo!, it has been a natural platform for Extract, Transform, and Load (ETL) and other forms of data pipelines. Without a mature workflow management and scheduling system, implementing such pipelines can be a challenge. Oozie satisfies these requirements and provides a viable tool to implement complex, real-world data pipelines. In this book, we have tried our best to introduce readers to all the facets of Oozie and walk them through the intricacies of this rather powerful and flexible platform.
Software workflow systems are ubiquitous and each system has its own idiosyncrasies. But Oozie is a lot more than just another workflow system. One of Oozie’s strengths is that it was custom built from the ground up for Hadoop. This not only means that Oozie works well on Hadoop, but that the authors of Oozie had an opportunity to build a new system incorporating much of their knowledge about other legacy workflow systems. Although some users view Oozie as just a workflow system, it has evolved into something more than that. The ability to use data availability and time-based triggers to schedule workflows via the Oozie coordinator is as important to today’s users as the workflow. ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access