Scala and Spark for Big Data Analytics

UnionRDD

UnionRDD is the result of a union operation of two RDDs. Union simply creates an RDD with elements from both RDDs as shown in the following code snippet:

class UnionRDD[T: ClassTag]( sc: SparkContext, var rdds: Seq[RDD[T]]) extends RDD[T](sc, Nil)

The following code snippet is the API call to create a UnionRDD by combining the elements of the two RDDs:

scala> val rdd_one = sc.parallelize(Seq(1,2,3))rdd_one: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[85] at parallelize at <console>:25scala> val rdd_two = sc.parallelize(Seq(4,5,6))rdd_two: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[86] at parallelize at <console>:25scala> val rdd_one = sc.parallelize(Seq(1,2,3))rdd_one: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[87] ...