CHAPTER 2   

Be Intentional about Risk to Humans

You should be protected from unsafe or ineffective systems.

—Blueprint for an AI Bill of Rights

On the night of September 26, 1983, a 44-year-old Soviet Air Defense Forces lieutenant-colonel named Stanislav Petrov was on duty monitoring a Soviet satellite-based missile tracking system when an alarm sounded. The system showed that five American intercontinental ballistic missiles were heading toward the Soviet Union from the United States. Petrov was trained to report the warning signal immediately to his commanding officer. Instead, he responded to his own “gut instinct,” as he later put it, and sent a report saying the alarm was a system malfunction.

Petrov considered the detection a computer ...

Get The AI Dilemma now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.