Frank Kane's Taming Big Data with Apache Spark and Python

What is the RDD?

Let's talk about the RDD in a reverse order because I'm weird like that. So, fundamentally, the RDD is a dataset, and it is an abstraction for a giant set of data, which is the main thing you need to know as a developer. What you'll do is to set up RDD objects and the RDD will load them up with big datasets and then call various methods on the RDD objects to distribute the processing of that data. Now the beauty is that although RDDs are both "distributed" and "resilient," you don't really have to worry about those things. RDDs can be spread out across an entire cluster of computers that may or may not be running locally, they can also handle the failure of specific executor nodes in your cluster automatically, keep on going ...