The Framework for ML Governance

Chapter 3. Governing ML During the Delivery and Operations Stages

Governing the delivery and operations stages of the lifecycle is the most difficult part of a comprehensive ML governance strategy (Figure 3-1). During development, there was an expected margin of error and a low risk of failure. It doesn’t matter if your data scientists make statistical mistakes along the way, as long as the mistakes are ironed out. Production does not grant the same lenience. Inadequate governance can easily lead to lapses in revenue, wasted resources, and even potentially irreparable damages to the company.

Figure 3-1. ML governance of delivery and operations

MLOps is the sum of production considerations for a machine learning model or pipeline. Again, it is a set of best practices for the technical and process-oriented portions of an ML pipeline delivering ML to production. During the delivery stage, data scientist responsibilities are phasing out while software, infrastructure, IT, and security responsibilities phase in. This stage has a massive bearing on how fast models get to production, and the value they can generate once they reach the final stage of operations. Operations includes at a minimum:¹

• Applications (software and business-facing)

• Application integrations

• Business intelligence (BI) dashboards and visualization

• Model management

• CI/CD (DevOps and MLOps)

• Infrastructure ...