book

Building Recommendation Systems in Python and JAX

by Bryan Bischof, Hector Yee

December 2023

Intermediate to advanced

338 pages

8h 57m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Conventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Key Components of a Recommendation SystemCollectorRankerServerSimplest Possible RecommendersThe Trivial RecommenderMost-Popular-Item RecommenderA Gentle Introduction to JAXBasic Types, Initialization, and ImmutabilityIndexing and SlicingBroadcastingRandom NumbersJust-in-Time CompilationSummary
The User-Item MatrixUser-User Versus Item-Item Collaborative FilteringThe Netflix ChallengeSoft RatingsData Collection and User LoggingWhat to LogCollection and InstrumentationFunnelsBusiness Insight and What People LikeSummary
Zipf’s Laws in RecSys and the Matthew EffectSparsityUser Similarity for Collaborative FilteringPearson CorrelationRatings via SimilarityExplore-Exploit as a Recommendation System ϵ -greedyWhat Should ϵ Be?The NLP-RecSys RelationshipVector SearchNearest-Neighbors SearchSummary
Online Versus OfflineCollectorOffline CollectorOnline CollectorRankerOffline RankerOnline RankerServerOffline ServerOnline ServerSummary
Revision Control SoftwarePython Build SystemsRandom-Item RecommenderObtaining the STL Dataset ImagesConvolutional Neural Network DefinitionModel Training in JAX, Flax, and OptaxInput PipelineSummary
Hydrating Your SystemPySparkExample: User Similarity in PySparkDataLoadersDatabase SnapshotsData Structures for Learning and InferenceVector SearchApproximate Nearest NeighborsBloom FiltersFun Aside: Bloom Filters as the Recommendation SystemFeature StoresSummary
Architectures by Recommendation StructureItem-to-User RecommendationsQuery-Based RecommendationsContext-Based RecommendationsSequence-Based RecommendationsWhy Bother with Extra Features?Encoder Architectures and Cold StartingDeploymentModels as APIsSpinning Up a Model ServiceWorkflow OrchestrationAlerting and MonitoringSchemas and PriorsIntegration TestsObservabilityEvaluation in ProductionSlow FeedbackModel MetricsContinuous Training and DeploymentModel DriftDeployment TopologiesThe Evaluation FlywheelDaily Warm StartsLambda Architecture and OrchestrationLoggingActive LearningSummary

Tech StackData RepresentationBig Data FrameworksCluster FrameworksPySpark ExampleGloVE Model DefinitionGloVE Model Specification in JAX and FlaxGloVE Model Training with OptaxSummary
Bilinear Factor Models (Metric Learning)Feature-Based Warm StartingSegmentation Models and HybridsTag-Based RecommendersHybridizationLimitations of Bilinear ModelsCounting RecommendersReturn to the Most-Popular-Item RecommenderCorrelation MiningPointwise Mutual Information via Co-occurrencesSimilarity from Co-occurrenceSimilarity-Based RecommendationsSummary
Latent SpacesDot Product SimilarityCo-occurrence ModelsReducing the Rank of a Recommender ProblemOptimizing for MF with ALSRegularization for MFRegularized MF ImplementationWSABIEDimension ReductionIsometric EmbeddingsNonlinear Locally Metrizable EmbeddingsCentered Kernel AlignmentAffinity and p-salePropensity Weighting for Recommendation System EvaluationPropensitySimpson’s and Mitigating ConfoundingSummary
EnvironmentsOnline and OfflineUser Versus Item MetricsA/B TestingRecall and Precision@ kPrecision at kRecall at kR-precisionmAP, MMR, NDCGmAPMRRNDCGmAP Versus NDCG?Correlation CoefficientsRMSE from AffinityIntegral Forms: AUC and cAUCRecommendation Probabilities to AUC-ROCComparison to Other MetricsBPRSummary
Where Does Ranking Fit in Recommender Systems?Learning to RankTraining an LTR ModelClassification for RankingRegression for RankingClassification and Regression for RankingWARPk-order StatisticBM25Multimodal RetrievalSummary
Experimentation TipsKeep It SimpleDebug Print StatementsDefer OptimizationKeep Track of ChangesUse Feature EngineeringUnderstand Metrics Versus Business MetricsPerform Rapid IterationSpotify Million Playlist DatasetBuilding URI DictionariesBuilding the Training DataReading the InputModeling the ProblemFraming the Loss FunctionExercisesSummary
Hard RankingLearned AvoidsHand-Tuned WeightsInventory HealthImplementing AvoidsModel-Based AvoidsSummary
Diversification of RecommendationsImproving DiversityApplying Portfolio OptimizationMultiobjective FunctionsPredicate PushdownFairnessSummary
ShardingLocality Sensitive Hashingk-d TreesHierarchical k-meansCheaper Retrieval MethodsSummary
Markov ChainsOrder-Two Markov ChainOther Markov ModelsRNN and CNN ArchitecturesAttention ArchitecturesSelf-Attentive Sequential RecommendationBERT4RecRecency SamplingMerging Static and SequentialSummary
Multimodal RecommendationsGraph-Based RecommendersNeural Message PassingApplicationsRandom WalksMetapath and HeterogeneityLLM ApplicationsLLM RecommendersLLM TrainingInstruct Tuning for RecommendationsLLM RankersRecommendations for AISummary

Content preview from Building Recommendation Systems in Python and JAX

Part I. Warming Up

How do we get all the data in the right place to train a recommendation system, and for real-time inference?

So, you’ve decided to dive into the world of recommendation systems! Are you hoping to suggest just the right thing based on users’ quirky preferences across a vast sea of choices? If so, you’ve set quite the challenge for yourself! On the surface, these systems might seem straightforward: if User A and User B have similar tastes, then maybe what A likes, B will too. But, as with all things that seem simple, there’s a depth that’s waiting to be explored.

How do we capture the essence of a user’s history and feed it into a model? Where do we stash this model so it’s ready to serve up suggestions on the fly? And how do we make sure it doesn’t suggest something that steps out of bounds or goes against the business rulebook? Collaborative filtering is our starting point, a guiding light. But there’s an entire universe beyond it that makes these systems tick, and together, we’re going to navigate it.