Skip to Content
View all events

Agentic AI Safety and Security

Published by O'Reilly Media, Inc.

Intermediate content levelIntermediate

Planning and Mitigating Safe AI Systems

Course Outcomes:

  • Design the role of context and larger systems in the safe usage of agentic AI
  • Evaluate concepts of ground truth and trust in the usage of AI systems
  • Identify risks and hazards in an agentic AI use case, regardless of the API protocol

Course Description

Agentic AI (and AI in general) has excited and concerned the public, organizational leaders, and politicians alike. There has been no invention quite like generative AI, especially when it can assist in tasks. They can even perform tasks with no humans in the loop thanks to breakthroughs in interfacing with APIs like MCP and A2A. This allows AI to connect to our existing systems and automate like never before. But with any new technology, blindsides and risks are plentiful. Thankfully, systems-oriented thinking can help us identify these risks and hazards, and break out of the task-oriented mindset that has become pervasive in the “move fast, break things” culture.

We will learn fundamental concepts of ground truth and how erroneous actions from an agentic AI system can propagate in a larger system. We discuss concepts of trustworthiness and operating domain, and how these two aspects go hand-in-hand designing safe and reliable systems that involve an AI. There will be case studies and practice identifying low risk and high risk applications, as well as discussion of data privacy, sourcing, and security issues. Finally, we will cover larger issues and how truly effective change and mitigation starts with us.

What you’ll learn and how you can apply it

  • Be able to systematically identify/address hazards in agentic applications
  • Effectively scope and frame how agentic AI is used for a given application
  • Speak to the risks of AI applications and create effective organizational policy

This live event is for you because...

  • You’re a machine learning practitioner (software developer, project manager) considering how to safely use AI in your own workflows
  • You work with executives who are either gung ho or leery of AI agency, and both need grounded understanding of what they can/cannot do
  • You are a manager in some capacity and want to understand generative AI opportunities and hazards, so you can inform and create productive dialogue with your workplace.

Prerequisites

  • Experience using an AI assistant like ChatGPT, Grok, Gemini, or Claude will be helpful to frame context.
  • Python proficiency is helpful, but not required.

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Intro (5 minutes)

A Brief Explanation of Agentic AI (30 minutes)

  • Presentation: Generative AI, LLMs, and AI Agents
  • Presentation: What is safety and security?
  • Presentation: How do AI agent systems work?
  • Presentation: API standards (MCP, A2A, etc)
  • LAB: Some Python examples of AI Agents
  • Hands-on exercise: Building a politeness social media bot

Trust (20 minutes)

  • Discussion: Can an AI agent ever be trusted to be correct or optimal?
  • Presentation: Practical verification and human-in-the-loop
  • Presentation: Why benchmarks can be irrelevant
  • Presentation: The problem of hallucination
  • Presentation: The problem of data drift
  • Pulse Check and Q&A
  • Exercise: Trust but verify
  • BREAK: 10 minutes

Operating Domain (10 minutes)

  • Discussion: How does scope affect an AI agent’s surface?
  • Discussion: What data does an AI agent ingest? What actions can it output?
  • Presentation: Importance of context and operating domain
  • Presentation: Task versus System-oriented thinking
  • Q&A
  • Exercise: AI Paralegal

Mapping Hazards and Risks (15 minutes)

  • Presentation: What is the AI agent being used for?
  • Discussion: What makes an AI system unsafe?
  • Presentation: Error propagation in a larger system
  • Presentation: Containing the operating domain
  • Presentation: Containing the human factor
  • Q&A
  • Exercise: What are the hazards and what can mitigate them?

Good and Bad Use Cases (20 minutes)

  • Discussion: What are some safe/unsafe cases for using an AI agent?
  • Discussion: Can an unsafe application be rescoped to become safe?
  • Poll and discussion | Code learning assistant
  • Poll and discussion | Anti-scammer bot
  • Poll and discussion | Email rewrite assistant
  • Poll and discussion | Creative writing
  • Poll and discussion | AI lawyer
  • Poll and discussion | AI BFF
  • Poll and discussion | AI Chef
  • Poll and discussion | AI stock trader
  • Q&A
  • BREAK: 10 minutes

Broader Issues - Data sourcing and Privacy (10 minutes)

  • Discussion: Where does labeled data come from?
  • Presentation: AI is learning from humans
  • Presentation: Privacy concerns and personal data
  • Presentation: Click labor and data entry farms
  • Q&A
  • Exercise: What happens when humans stop providing data?
  • Hands-on exercise: Guitars and clocks

Broader Issues - AI Slop (10 minutes)

  • Discussion: What stories have you heard about slop and bots?
  • Presentation: Social media and AI-generated content
  • Presentation: Are we going to drown in slop and AI agent bots?
  • Q&A
  • Exercise: What are some technical solutions to slop on social media?

Broader Issues - Detecting AI-generated Content (10 minutes)

  • Presentation: The hard problem of using AI to detect AI
  • Presentation: Can we adapt?
  • Q&A

Broader Issues - The economics of hype (17 minutes)

  • Discussion: The AI Scapegoat
  • Discussion: Is AI replacing workers?
  • Presentation: The importance of incentive structures
  • Exercise: Show me the incentive, and I’ll show you the outcome

Broader Issues - Truly Effective Change (13 minutes)

  • Discussion: Regulate! But how?
  • Presentation: Why effective policy is so elusive.
  • Presentation: Unintended consequences and conflicting interests
  • Presentation: Effective change | Responsible company policy
  • Presentation: Effective change | Educating the public, not just lawmakers
  • Presentation: Effective change | Engage and mobilize the community

Your Instructor

  • Thomas Nield

    Thomas Nield is the founder of Nield Consulting Group and an instructor at University of Southern California, where he teaches AI System Safety, developing systematic approaches for identifying AI-related hazards in aviation and ground vehicles. He's authored three books, including Essential Math for Data Science and Getting Started with SQL (both for O'Reilly). He enjoys making technical content relatable and relevant to those unfamiliar or intimidated by it. Thomas teaches classes on data analysis, machine learning, mathematical optimization, and practical artificial intelligence. He’s also the founder and inventor of Yawman Flight, a company that develops universal handheld flight controls for flight simulation and unmanned aerial vehicles.

    Xlinksearch

Skill covered

AI Security