Beginning Apache Spark 3: With DataFrame, Spark SQL, Structured Streaming, and Spark Machine Learning Library

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2021

H. LuuBeginning Apache Spark 3https://doi.org/10.1007/978-1-4842-7383-8_6

6. Spark Streaming

Hien Luu¹  

(1)

SAN JOSE, CA, USA

 

In addition to batch data processing, stream processing has become a must-have capability for any business to harness the value of real-time data to increase their competitive advantages, make better business decisions, or improve user experience. With the advent of the Internet of Things, the volume and velocity of real-time data has increased. For Internet companies like Facebook, LinkedIn, or Twitter, millions of social activities happening every second on their platform are represented as streaming data.

At a high level, stream processing ...