Beginning Apache Spark 3: With DataFrame, Spark SQL, Structured Streaming, and Spark Machine Learning Library

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2021

H. LuuBeginning Apache Spark 3https://doi.org/10.1007/978-1-4842-7383-8_7

7. Advanced Spark Streaming

Hien Luu¹  

(1)

SAN JOSE, CA, USA

 

Chapter 6 introduced the core concepts in streaming processing, the Spark Structured Streaming processing engine’s features, and the basic steps of developing a streaming application. Real-world streaming applications usually need to extract insights or patterns from the incoming real-time data at scale and feed that information into downstream applications to make business decisions or save that information in some storage system for further analysis or visualization purposes. Another aspect of real-world streaming applications ...