book

Building Recommendation Systems in Python and JAX

by Bryan Bischof, Hector Yee

December 2023

Intermediate to advanced

338 pages

8h 57m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Conventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Key Components of a Recommendation SystemCollectorRankerServerSimplest Possible RecommendersThe Trivial RecommenderMost-Popular-Item RecommenderA Gentle Introduction to JAXBasic Types, Initialization, and ImmutabilityIndexing and SlicingBroadcastingRandom NumbersJust-in-Time CompilationSummary
The User-Item MatrixUser-User Versus Item-Item Collaborative FilteringThe Netflix ChallengeSoft RatingsData Collection and User LoggingWhat to LogCollection and InstrumentationFunnelsBusiness Insight and What People LikeSummary
Zipf’s Laws in RecSys and the Matthew EffectSparsityUser Similarity for Collaborative FilteringPearson CorrelationRatings via SimilarityExplore-Exploit as a Recommendation System ϵ -greedyWhat Should ϵ Be?The NLP-RecSys RelationshipVector SearchNearest-Neighbors SearchSummary
Online Versus OfflineCollectorOffline CollectorOnline CollectorRankerOffline RankerOnline RankerServerOffline ServerOnline ServerSummary
Revision Control SoftwarePython Build SystemsRandom-Item RecommenderObtaining the STL Dataset ImagesConvolutional Neural Network DefinitionModel Training in JAX, Flax, and OptaxInput PipelineSummary
Hydrating Your SystemPySparkExample: User Similarity in PySparkDataLoadersDatabase SnapshotsData Structures for Learning and InferenceVector SearchApproximate Nearest NeighborsBloom FiltersFun Aside: Bloom Filters as the Recommendation SystemFeature StoresSummary
Architectures by Recommendation StructureItem-to-User RecommendationsQuery-Based RecommendationsContext-Based RecommendationsSequence-Based RecommendationsWhy Bother with Extra Features?Encoder Architectures and Cold StartingDeploymentModels as APIsSpinning Up a Model ServiceWorkflow OrchestrationAlerting and MonitoringSchemas and PriorsIntegration TestsObservabilityEvaluation in ProductionSlow FeedbackModel MetricsContinuous Training and DeploymentModel DriftDeployment TopologiesThe Evaluation FlywheelDaily Warm StartsLambda Architecture and OrchestrationLoggingActive LearningSummary

Tech StackData RepresentationBig Data FrameworksCluster FrameworksPySpark ExampleGloVE Model DefinitionGloVE Model Specification in JAX and FlaxGloVE Model Training with OptaxSummary
Bilinear Factor Models (Metric Learning)Feature-Based Warm StartingSegmentation Models and HybridsTag-Based RecommendersHybridizationLimitations of Bilinear ModelsCounting RecommendersReturn to the Most-Popular-Item RecommenderCorrelation MiningPointwise Mutual Information via Co-occurrencesSimilarity from Co-occurrenceSimilarity-Based RecommendationsSummary
Latent SpacesDot Product SimilarityCo-occurrence ModelsReducing the Rank of a Recommender ProblemOptimizing for MF with ALSRegularization for MFRegularized MF ImplementationWSABIEDimension ReductionIsometric EmbeddingsNonlinear Locally Metrizable EmbeddingsCentered Kernel AlignmentAffinity and p-salePropensity Weighting for Recommendation System EvaluationPropensitySimpson’s and Mitigating ConfoundingSummary
EnvironmentsOnline and OfflineUser Versus Item MetricsA/B TestingRecall and Precision@ kPrecision at kRecall at kR-precisionmAP, MMR, NDCGmAPMRRNDCGmAP Versus NDCG?Correlation CoefficientsRMSE from AffinityIntegral Forms: AUC and cAUCRecommendation Probabilities to AUC-ROCComparison to Other MetricsBPRSummary
Where Does Ranking Fit in Recommender Systems?Learning to RankTraining an LTR ModelClassification for RankingRegression for RankingClassification and Regression for RankingWARPk-order StatisticBM25Multimodal RetrievalSummary
Experimentation TipsKeep It SimpleDebug Print StatementsDefer OptimizationKeep Track of ChangesUse Feature EngineeringUnderstand Metrics Versus Business MetricsPerform Rapid IterationSpotify Million Playlist DatasetBuilding URI DictionariesBuilding the Training DataReading the InputModeling the ProblemFraming the Loss FunctionExercisesSummary
Hard RankingLearned AvoidsHand-Tuned WeightsInventory HealthImplementing AvoidsModel-Based AvoidsSummary
Diversification of RecommendationsImproving DiversityApplying Portfolio OptimizationMultiobjective FunctionsPredicate PushdownFairnessSummary
ShardingLocality Sensitive Hashingk-d TreesHierarchical k-meansCheaper Retrieval MethodsSummary
Markov ChainsOrder-Two Markov ChainOther Markov ModelsRNN and CNN ArchitecturesAttention ArchitecturesSelf-Attentive Sequential RecommendationBERT4RecRecency SamplingMerging Static and SequentialSummary
Multimodal RecommendationsGraph-Based RecommendersNeural Message PassingApplicationsRandom WalksMetapath and HeterogeneityLLM ApplicationsLLM RecommendersLLM TrainingInstruct Tuning for RecommendationsLLM RankersRecommendations for AISummary

Content preview from Building Recommendation Systems in Python and JAX

Chapter 9. Feature-Based and Counting-Based Recommendations

Consider this oversimplified problem: given a bunch of new users, predict which will like our new mega-ultra-fancy-fun-item-of-novelty, or MUFFIN for short. You may start by asking which old users like MUFFIN; do those users have any aspects in common? If so, you could build a model that predicts MUFFIN affinity from those correlated user features.

Alternatively, you could ask, “What are other items people buy with MUFFIN?” If you find that others frequently also ask for JAM (just-awesome-merch), then MUFFIN may be a good suggestion for those who already have JAM. This would be using the co-occurrence of MUFFIN and JAM as a predictor. Similarly, if your friend comes along with tastes similar to yours—you both like SCONE, JAM, BISCUIT, and TEA—but your friend hasn’t yet had the MUFFIN, if you like MUFFIN, it’s probably a good choice for your friend too. This is using the co-occurrence of items between you and your friend.

These item relationship features will form our first ranking methods in this chapter; so grab a tasty snack and let’s dig in.

Bilinear Factor Models (Metric Learning)

As per the usual idioms about running in front of horses and walking after the cart, let’s start our journey into ranking systems with what can be considered the naive ML approaches. Via these approaches, we will start to get a sense of where the rub lies in building recommendation systems and why some of the forthcoming efforts are necessary ...