Learning Apache Apex
by Ananth Gundabattula, Thomas Weise, Munagala V. Ramanath, David Yan, Kenneth Knowles
Summary
In this chapter, we examined a powerful tool in Apex library—the SQL API (Calcite integration)—and how it enables us to build classical ETL applications using SQL to automatically create many operators and link them into the DAG. We also covered how to build the application to produce the application archive (.apa) file which can be deployed in a cluster.
We also saw how to run the integration test locally, on our development machine, to enable detection of bugs, configuration errors, and other defects early in the development process, without the need to install Hadoop, ZooKeeper, or Kafka. We then took a detailed look at the application log messages and saw how to interpret them.
We rounded out the chapter by summarizing the classes ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access