Chapter 18AI Threat Modeling and Attack Surfaces

Artificial intelligence (AI) has redefined how organizations operate, but it has also reshaped how auditors must think about assurance, control, and accountability. The velocity of AI adoption—spanning models, datasets, prompts, and automated agents—demands a disciplined approach to understanding where risks emerge and how they can be mitigated within auditable boundaries. This chapter positions the auditor at the intersection of automation, analytics, and adaptive systems, equipping them to evaluate AI not as a black box but as an integrated, evolving ecosystem. Understanding this ecosystem is essential to maintaining trust, transparency, and regulatory compliance in environments where decisions are increasingly delegated to machine-driven logic.
From threat modeling to control design, AI auditing extends traditional assurance practices into domains where data, logic, and behavior are constantly in flux. This chapter examines how familiar frameworks can be adapted to map AI-specific attack surfaces—prompt injections, data poisoning, model theft, and misuse through integrations—and how these threats translate into measurable control objectives across identity, data protection, and behavioral safety. By emphasizing data integrity, traceability, and governance, it establishes a foundation for evidence-driven assurance that can withstand ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access