Frank Kane's Taming Big Data with Apache Spark and Python

Making item-based collaborative filtering a Spark problem

Turning item-based collaborative filtering into a Spark problem is complicated. I'm just going to kind of walk you through the approach that I used. The point here is just to illustrate how you can take something like this and turn it into a Spark problem. It's just an example to look at, so if you don't totally get everything that's going on here that's okay, the important thing is that you see that you can take a complex problem like this and make a Spark problem out of it and make it actually work. So here's our strategy:

• Map input ratings to (userID, (movieID, rating)):
  • We map the set of input ratings from the MovieLens dataset to key/value pairs, where the key is the user ID ...