Structured Streaming
Streaming is a seemingly broad topic! If you take a closer look at the real-world problems, businesses do not just want a streaming engine to make decisions in real time. There has always been a need to integrate both batch stack and streaming stack, and integrate with external storage systems and applications. Also, the solution should be such that it should adapt to dynamic changes in business logic to address new and changing business requirements.
Apache Spark 2.0 has the first version of the higher level stream processing API called the Structured Streaming engine. This scalable and fault-tolerant engine leans on the Spark SQL API to simplify the development of real-time, continuous big data applications. It is probably ...
Get Spark for Data Science now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
