Mastering Spark for Data Science

Building a recommender

Now that we've explored our song analyzer, let's get back on track with the recommendation engine. As discussed earlier, we would like to recommend songs based on frequency hashes extracted from audio signals. Taking as an example the dispute between Led Zeppelin and Spirit, we would expect both songs to be relatively close to each other, as the allegation is that they share a melody. Using this thought as our main assumption, we could potentially recommend Taurus to someone interested in Stairway to Heaven.

The PageRank algorithm

Instead of recommending a specific song, we will recommend playlists. A playlist would consist of a list of all our songs ranked by relevance, most to least relevant. Let's begin with the assumption ...