Stream Processing with Apache Flink

Chapter 11. Where to Go from Here?

It has been a long journey and you have made it to the end of this book! But your Flink journey has just started, and this chapter points to the possible paths you can take from here. We will provide you with a brief tour of the additional Flink functionality not included in this book and give you some pointers to further Flink resources. There exists a vibrant community around Flink and we encourage you to connect with other users, start contributing, or find out what companies are building with Flink to help inspire your own work.

The Rest of the Flink Ecosystem

While this book is particularly focused on stream processing, Flink is in fact a general-purpose distributed data processing framework and can be used for other types of data analysis as well. Further, Flink offers domain-specific libraries and APIs for relational queries, complex event processing (CEP), and graph processing.

The DataSet API for Batch Processing

Flink is a full-fledged batch processor and can be used to implement use cases requiring one-off or periodic queries on bounded input data. DataSet programs are specified as a series of transformations just like DataStream programs with the difference that a DataSet is a bounded data collection. The DataSet API provides operators to perform filtering, mapping, selection, joins, and groupings, as well as connectors to read and write datasets from and to external systems, such as filesystems and databases. Using the DataSet ...