Frank Kane's Taming Big Data with Apache Spark and Python

Why did we get bad results?

These results are pretty sketchy, so let's talk about what might be going on there. The first thing to mention is there are a couple of parameters to that ALS method there for Alternating Least Squares and the results are very sensitive to the parameters you choose. It actually takes a lot of work to find the optimal parameters to get good recommendations out a given dataset. Now, there are some techniques you can use in machine learning to try to arrive at the right parameters. There's a technique called "train/test," where you withhold a portion of the data as sort of a training dataset and another portion as a testing dataset, and you evaluate your ability to predict user ratings using your algorithm going forward. ...