Not with a Bug, But with a Sticker

Appendix ABig-Picture Questions

10-701 is the registration code for the machine learning class at Carnegie Mellon University. I took this class purely at my graduate advisor's goading, who wanted me to “look beyond” computer security. I remember my advisor saying, “Tom wrote the textbook on ML, and you will pick up something from him; give the class a chance.” So, for two days a week for an entire semester in 2011, I filed into the always-chilly auditorium with 50 other students into Wean Hall, CMU's ugliest, brutalist building, to hear the ML virtuoso, Tom Mitchell. This class set in motion a decade-long obsession with combining computer security and machine learning.

In lieu of recapping all the lectures at the end of the semester, Mitchell shared the five big-picture questions about machine learning that he carries in his head. His rationale was that if you knew what questions to ask when confronted with a new ML algorithm, you could pretty much get its gist. Whenever I read a landmark paper in machine learning, these big-picture questions have played Virgil to my Dante.

In that spirit, Hyrum and I are providing five questions that we encourage business executives, policymakers, and engineering managers to ask teams building and deploying ML systems. On their face, these questions are not earth-shattering. But they serve as a useful tool to begin important conversations about your organization's security posture for ML systems.

As ML continues to proliferate, everyone—governments, ...