May 2017
Beginner to intermediate
596 pages
15h 2m
English
Content preview from Data Lake for Enterprises
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
DataSet API
Flink handles batch data and processes it using the Flink’s built-in DataSet API. The DataSet API also has mechanisms to do necessary transformations necessary for common use cases. After transformation, the DataSet is transformed to DataSet. API’s are in Java, Scala, and Python. In Flink batch data and its processing also work on the same aspect as stream data. The following figure shows basic working of the DataSet API on a batch data using the MapReduce paradigm producing DataSet (class of objects created when data is read and processed):
Figure 21: Flink DataSet API program execution