Scala and Spark for Big Data Analytics

Transformations

Transformations on a DStream are similar to the transformations applicable to a Spark core RDD. Since DStream consists of RDDs, a transformation also applies to each RDD to generate a transformed RDD for the RDD, and then a transformed DStream is created. Every transformation creates a specific DStream derived class.

The following diagram shows the hierarchy of DStream classes starting from the parent DStream class. We can also see the different classes inheriting from the parent class:

There are a lot of DStream classes purposely built for the functionality. Map transformations, window functions, reduce actions, and different ...