Building Recommendation Systems in Python and JAX

Chapter 13. Putting It All Together: Experimenting and Ranking

In the last few chapters, we have covered many aspects of ranking, including various kinds of loss functions as well as metrics for measuring the performance of ranking systems. In this chapter, we will show an example of a ranking loss and ranking metric on the Spotify Million Playlist dataset.

This chapter encourages a lot more experimentation and is more open-ended than the previous ones, whose goal was to introduce concepts and infrastructure. This chapter, on the other hand, is written to encourage you to roll up your sleeves and engage directly with loss functions and writing metrics.

Experimentation Tips

Before we begin digging into the data and modeling, let’s cover some practices that will make your life easier when doing a lot of experimentation and rapid iteration. These are general guidelines that have made our experimentation faster. As a result, we’re able to rapidly iterate toward solutions that help us reach our objectives.

Experimental code is different from engineering code in that the code is written to explore ideas, not for robustness. The goal is to achieve maximum velocity while not sacrificing too much in terms of code quality. So you should think about whether a piece of code should be thoroughly tested or whether this isn’t necessary because the code is present only to test a hypothesis and then it will be thrown away. With that in mind, here are some tips. Keep in mind that these tips are ...