book

Ten Things to Know About ModelOps

by Thomas Hill, Mark Palmer, Larry Derany

June 2022

Beginner to intermediate

41 pages

59m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Overview of ModelOps Data-Driven OrganizationsExtracting Information from Data via ModelsThe Model Building/Authoring Life CycleWhat Are ModelOps Tools?
1. ModelOps Is the Enterprise “Operating System” for AI/ML AlgorithmsMany Types of ModelsMany Different People/Roles Build Models2. There Are Many Algorithms and Many Model Formats: Deliver Them AllModel FormatsSupport Your Model Builders, Don’t Constrain Them3. First, Do No Harm: Anticipate Risks Rather Than React to Damage DoneOffending CustomersRisky Predictions for ManufacturingConsidering Wrong PredictionsAnticipating Risk4. Aim to Build Your Organization’s IP Secret Sauce Through Repeatable Processing StepsManaging and Accumulating IP Through ModelsDisorganized Analytics Across Silos5. ModelOps Is About Maintaining and Expanding Your IPModel Decay (Drift): Changing Populations, Data, and RelationshipsFlexible Monitoring of End-to-End Scoring Pipelines and Agile DeploymentThe Bane of AI/ML at Scale: Big CodeGoverned Inputs/Outputs; Small Reusable Steps; and Efficient, Targeted Monitoring That Enables Automation6. The World Is Changing Fast: Think Agile Model Deployment and Streaming DataBatch and Real-Time Data—Use CasesResponding to a Dynamically Changing World7. Plan for Success and Monitor for Value (Not Just Accuracy)Every Project Starts at the End—or It Is Likely to Fail!Planning for How Model Scoring Is to Be Consumed—and Valuable8. The Future of AI Will Be Regulated—ModelOps Helps Ensure ComplianceRequirements for Analytics in Medical Device and Pharma Manufacturing Offer Guidance for What’s to ComeModelOps Capabilities, Beyond Efficient Deployment and Scoring9. Expect Quality and Process Best Practices from the ModelOps Operating System10. ModelOps Is a Required Tool for Managing AI/ML Model Deployment: Start Small, but Start Now

Content preview from Ten Things to Know About ModelOps

Chapter 2. Ten Things to Know About ModelOps

1. ModelOps Is the Enterprise “Operating System” for AI/ML Algorithms

While AI/ML and deep learning methods get the most attention in the press, blogs, and on the web, the reality is that for the majority of applications the model authoring may be the simplest part. The supporting logic, processing, deployment, and controls—and in particular setting up an efficient delivery system of solutions infused by AI/ML—are the much bigger challenge.

ModelOps, therefore, is a kind of operating system for AI/ML models. That is, it creates a platform and set of tools to deploy algorithms in a reliable, repeatable, compliant, safe, and efficient way. Like the operating system on a laptop or mobile phone, it provides access to the right algorithm and to the right place at the right time when it is needed.

Many Types of Models

To reiterate, models are just transformations of data, in order to extract useful information. There are many types of models that can convey useful information, help make better decisions, or drive effective automation. Continuing with the operating system metaphor, it is a mistake to limit ModelOps to just one “app”: ModelOps tools must deploy AI algorithms built on any platform or provided by any cloud service.

Any statistical analysis, aggregation, set of rules, summary, or BI dashboard represents information, abstractions, and inferences from observed data that guide strategies and decisions. All of the above are “models,” ...