MLOps with Ray: Best Practices and Strategies for Adopting Machine Learning Operations

© Hien Luu, Max Pumperla and Zhe Zhang 2024

H. Luu et al.MLOps with Rayhttps://doi.org/10.1007/979-8-8688-0376-5_8

8. An Introduction to the Ray AI Libraries

Hien Luu¹  , Max Pumperla² and Zhe Zhang³

(1)

Santa Clara, CA, USA

(2)

Bad Segeberg, Germany

(3)

Sunnyvale, CA, USA

 

Overview

In the last chapter you’ve learned about Ray Core, and how it can be used to build distributed applications. This chapter introduces you to the main concepts of Ray’s Artificial Intelligence (AI) libraries and how you can utilize them for creating and deploying typical ML workflows. We will demonstrate how to use those libraries by building an application that fine-tunes a large language model (LLM), deploys it for online inference, and uses it for offline batch inference. We ...