MLOps with Ray: Best Practices and Strategies for Adopting Machine Learning Operations

© Hien Luu, Max Pumperla and Zhe Zhang 2024

H. Luu et al.MLOps with Rayhttps://doi.org/10.1007/979-8-8688-0376-5_5

5. Model Serving Infrastructure

Hien Luu¹  , Max Pumperla² and Zhe Zhang³

(1)

Santa Clara, CA, USA

(2)

Bad Segeberg, Germany

(3)

Sunnyvale, CA, USA

 

In the machine learning (ML) community, there is a common saying that the ROI of an ML project starts when the model is in production. This phrase reminds us that the true value of a ML project is realized when the trained model is deployed and actively used in production. The model serving infrastructure plays a crucial role in operationalizing ML models in production and integrating the ML projects into the operations of an organization, such as predicting customer churn, detecting fraudulent ...