Hands-On Machine Learning with C++

Assessing system quality

Testing a recommender system is a complicated process that always poses many questions, mainly due to the ambiguity of the concept of quality.

In general, in machine learning problems, there are the following two main approaches to testing:

• Offline model testing on historical data using retro tests
• Testing the model using A/B testing (we run several options, and see which one gives the best result)

Both of the preceding approaches are actively used in the development of recommender systems. The main limitation that we have to face is that we can evaluate the accuracy of the forecast only on those products that the user has already evaluated or rated. The standard approach is cross-validation, with the leave-one-out ...