Practical Real-time Data Processing and Analytics

Spark Streaming operations

Spark provides various operations that can be performed over DStreams. All operations are categorized into transformation and output operations. Let us discuss both these operations:

• Transformation operations: Transformations are those operations that help with modification or change the structure of data in input streams. They are similar and support almost all transformation operations provided by RDDs; for example, map(), flatmap(), union(), and many more. Refer to the DStream API at http://tinyurl.com/znfgb8a for a complete list of transformation operations supported on input streams. Apart from the regular transformation operations, as defined by DStreams and similar to RDD, DStreams provide a few special ...