Chapter 4. Computation: Designing for the Processing and Generation Phase
On July 20, 1969, during the final descent of Apollo 11 toward the Moon, a warning alarm lit up the astronauts’ display: 1202.
Unfortunately, no one in the cockpit knew what 1202 meant; the computer didn’t appear to be crashing, but other than that, it was a mystery. The alarm indicated something was wrong.
The 1202 alarm turned out to be a low-priority overload warning. The onboard computer was shedding nonessential tasks to stay focused on landing-critical ones—an intentional feature that hadn’t been fully thought through. From an engineering perspective, this was an elegant act of triage, not a malfunction. But to the astronauts, it looked like a potentially fatal glitch. They had no way to see how the system was allocating its computational resources, or why some functions were dropped while others were preserved. The system was working under strain, but because that logic was invisible, it became a source of confusion and near abort.
Encountering an error in AI can feel much the same. Many parts of the process feel confusing, locked behind occult signifiers, much like the 1202 alarm. Understanding this middle layer—how AI systems process, triage, and generate outputs—is essential for building interfaces that balance the abilities of AI with the real human needs of its users.
Once the input is submitted, the user expects an answer, but before an AI can respond, it has to compute. That might sound obvious, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access