Practical Real-time Data Processing and Analytics

RDD pragmatic exploration

We have read and understood well that RDDs are an immutable, distributed collection of object values used as a unit of abstraction in Spark Framework. There are two ways RDDs can be created:

• Loading external dataset
• Distributing a list/set/collection of objects in their driver program

Now let's create some simple programs to create and use RDDs:

The preceding screenshot captures quick steps to create an RDD on Spark Shell. Here are the specific commands and further transformational outputs for this:

Scala> val inputfile = sc.textFile("input.txt") 

The preceding command reads the file called input.txt from the ...