Distributed Data Systems with Azure Databricks

Chapter 11: Managing and Serving Models with MLflow and MLeap

In the previous chapter, we learned how we can fine-tune models created in Azure Databricks. The next step is how we can effectively keep track and make use of the models that we train. Software development has clear methodologies for keeping track of code, having stages such as staging or production versions of the code and general code lifecycle management processes, but it's not that common to see that applied to machine learning models. The reasons for this might vary, but one reason could be that the data science team follows its own methodologies that might be closer to academia than the production of software, as well as the fact that machine learning doesn't have clearly ...