Structured Streaming

Streaming is a seemingly broad topic! If you take a closer look at the real-world problems, businesses do not just want a streaming engine to make decisions in real time. There has always been a need to integrate both batch stack and streaming stack, and integrate with external storage systems and applications. Also, the solution should be such that it should adapt to dynamic changes in business logic to address new and changing business requirements.

Apache Spark 2.0 has the first version of the higher level stream processing API called the Structured Streaming engine. This scalable and fault-tolerant engine leans on the Spark SQL API to simplify the development of real-time, continuous big data applications. It is probably ...

Get Spark for Data Science now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.