book

Building Recommendation Systems in Python and JAX

by Bryan Bischof, Hector Yee

December 2023

Intermediate to advanced

338 pages

8h 57m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Conventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Key Components of a Recommendation SystemCollectorRankerServerSimplest Possible RecommendersThe Trivial RecommenderMost-Popular-Item RecommenderA Gentle Introduction to JAXBasic Types, Initialization, and ImmutabilityIndexing and SlicingBroadcastingRandom NumbersJust-in-Time CompilationSummary
The User-Item MatrixUser-User Versus Item-Item Collaborative FilteringThe Netflix ChallengeSoft RatingsData Collection and User LoggingWhat to LogCollection and InstrumentationFunnelsBusiness Insight and What People LikeSummary
Zipf’s Laws in RecSys and the Matthew EffectSparsityUser Similarity for Collaborative FilteringPearson CorrelationRatings via SimilarityExplore-Exploit as a Recommendation System ϵ -greedyWhat Should ϵ Be?The NLP-RecSys RelationshipVector SearchNearest-Neighbors SearchSummary
Online Versus OfflineCollectorOffline CollectorOnline CollectorRankerOffline RankerOnline RankerServerOffline ServerOnline ServerSummary
Revision Control SoftwarePython Build SystemsRandom-Item RecommenderObtaining the STL Dataset ImagesConvolutional Neural Network DefinitionModel Training in JAX, Flax, and OptaxInput PipelineSummary
Hydrating Your SystemPySparkExample: User Similarity in PySparkDataLoadersDatabase SnapshotsData Structures for Learning and InferenceVector SearchApproximate Nearest NeighborsBloom FiltersFun Aside: Bloom Filters as the Recommendation SystemFeature StoresSummary
Architectures by Recommendation StructureItem-to-User RecommendationsQuery-Based RecommendationsContext-Based RecommendationsSequence-Based RecommendationsWhy Bother with Extra Features?Encoder Architectures and Cold StartingDeploymentModels as APIsSpinning Up a Model ServiceWorkflow OrchestrationAlerting and MonitoringSchemas and PriorsIntegration TestsObservabilityEvaluation in ProductionSlow FeedbackModel MetricsContinuous Training and DeploymentModel DriftDeployment TopologiesThe Evaluation FlywheelDaily Warm StartsLambda Architecture and OrchestrationLoggingActive LearningSummary

Tech StackData RepresentationBig Data FrameworksCluster FrameworksPySpark ExampleGloVE Model DefinitionGloVE Model Specification in JAX and FlaxGloVE Model Training with OptaxSummary
Bilinear Factor Models (Metric Learning)Feature-Based Warm StartingSegmentation Models and HybridsTag-Based RecommendersHybridizationLimitations of Bilinear ModelsCounting RecommendersReturn to the Most-Popular-Item RecommenderCorrelation MiningPointwise Mutual Information via Co-occurrencesSimilarity from Co-occurrenceSimilarity-Based RecommendationsSummary
Latent SpacesDot Product SimilarityCo-occurrence ModelsReducing the Rank of a Recommender ProblemOptimizing for MF with ALSRegularization for MFRegularized MF ImplementationWSABIEDimension ReductionIsometric EmbeddingsNonlinear Locally Metrizable EmbeddingsCentered Kernel AlignmentAffinity and p-salePropensity Weighting for Recommendation System EvaluationPropensitySimpson’s and Mitigating ConfoundingSummary
EnvironmentsOnline and OfflineUser Versus Item MetricsA/B TestingRecall and Precision@ kPrecision at kRecall at kR-precisionmAP, MMR, NDCGmAPMRRNDCGmAP Versus NDCG?Correlation CoefficientsRMSE from AffinityIntegral Forms: AUC and cAUCRecommendation Probabilities to AUC-ROCComparison to Other MetricsBPRSummary
Where Does Ranking Fit in Recommender Systems?Learning to RankTraining an LTR ModelClassification for RankingRegression for RankingClassification and Regression for RankingWARPk-order StatisticBM25Multimodal RetrievalSummary
Experimentation TipsKeep It SimpleDebug Print StatementsDefer OptimizationKeep Track of ChangesUse Feature EngineeringUnderstand Metrics Versus Business MetricsPerform Rapid IterationSpotify Million Playlist DatasetBuilding URI DictionariesBuilding the Training DataReading the InputModeling the ProblemFraming the Loss FunctionExercisesSummary
Hard RankingLearned AvoidsHand-Tuned WeightsInventory HealthImplementing AvoidsModel-Based AvoidsSummary
Diversification of RecommendationsImproving DiversityApplying Portfolio OptimizationMultiobjective FunctionsPredicate PushdownFairnessSummary
ShardingLocality Sensitive Hashingk-d TreesHierarchical k-meansCheaper Retrieval MethodsSummary
Markov ChainsOrder-Two Markov ChainOther Markov ModelsRNN and CNN ArchitecturesAttention ArchitecturesSelf-Attentive Sequential RecommendationBERT4RecRecency SamplingMerging Static and SequentialSummary
Multimodal RecommendationsGraph-Based RecommendersNeural Message PassingApplicationsRandom WalksMetapath and HeterogeneityLLM ApplicationsLLM RecommendersLLM TrainingInstruct Tuning for RecommendationsLLM RankersRecommendations for AISummary

Content preview from Building Recommendation Systems in Python and JAX

About the Authors

Bryan Bischof leads AI at Hex and is an adjunct professor in the Rutgers University Business Analytics master’s program, where he teaches data science. Previously, he was the head of data science at Weights & Biases and built the DS, ML, and data engineering teams. He has built recommendation systems for clothing (at Stitch Fix), recommendation systems for technical blog posts (at Weights & Biases), and the world’s first recommendation system for coffee (at Blue Bottle Coffee); currently, he is building recommendation systems for AI agents. His research interests are in geometric methods for ML, including higher-order graph methods and topological methods. His data visualization work appears in the popular book The Day It Finally Happens by Mike Pearl. His Ph.D. is in pure mathematics. You can find Bryan on LinkedIn at bryan-bischof and on Twitter at @bebischof.

Hector Yee is a staff software engineer at Google, where he has worked on multiple projects including creating the first content-based ranker on Image Search and self driving car perception. He also worked on the YouTube recommender system and was part of the team that won a technical Emmy Award for their work on personalized video ranking technology. He has an M.S. in computer graphics. You can find Hector on LinkedIn at yeehector and on Twitter at @eigenhector.