11 Model measurement and why it’s so important

This chapter covers

  • Methodologies for determining the impact of a model
  • A/B testing approaches for attribution data collection

In part 1, we focused on aligning ML project work to business problems. This is, after all, the most critical aspect for making the solution viable. While those earlier chapters focused on communication before, during, and immediately up to a production release, this chapter focuses on the project communication after release. We’ll cover how to present, discuss, and accurately report on the long-term health of ML projects—specifically, in language and methodologies that the business will understand.

Discussions about model performance are complex. While the business is ...

Get Machine Learning Engineering in Action now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.