Beginning Apache Spark 3: With DataFrame, Spark SQL, Structured Streaming, and Spark Machine Learning Library

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2021

H. LuuBeginning Apache Spark 3https://doi.org/10.1007/978-1-4842-7383-8_9

9. Managing the Machine Learning Life Cycle

Hien Luu¹  

(1)

SAN JOSE, CA, USA

 

As companies leverage AI and machine learning to transform their business, they soon realize developing and deploying ML applications is not a small task. In Chapter 8, you learned that the machine learning development process is a highly iterative and scientific process that needs an engineering culture and practice that is slightly different from the traditional software development process. As the machine learning development community, including data scientist, *ML engineers and software engineers, gains ...