Skip to Content
View all events

AI-Native Software Architecture

Published by O'Reilly Media, Inc.

Intermediate content levelIntermediate

Design reliable LLM systems as end-to-end pipelines

What you’ll learn and how you can apply it

  • Design LLM applications as pipeline-based systems rather than single model calls
  • Implement prompt, retrieval, and governance layers to reduce failure modes
  • Apply validation, tracing, and evaluation techniques to improve system reliability
  • Compose AI-native architecture patterns into a production-ready request flow

Course description

Modern applications built on large language models often fail in production, not because the models are weak but because they’re wrapped in architectures designed for deterministic systems. Teams treat LLMs as single API calls instead of probabilistic systems that require structure, validation, and feedback loops.

Join Sujata Sridharan and Vrinda Bhatia to discover how to design AI-native software architecture by treating LLM applications as end-to-end pipelines. You’ll implement a practical, four-layer approach covering input framing, information retrieval, decisioning and fallbacks, and output validation and monitoring. Through guided exercises based on lessons drawn from real production systems, you’ll apply architectural patterns that improve reliability, observability, and predictability. By the end of the course, you’ll have a working reference implementation and a clear framework you can adapt to your own AI-powered applications.

This live event is for you because...

  • You’re a software engineer, senior engineer, or tech lead.
  • You work with LLM-backed or AI-enabled production systems.
  • You want to become an engineer who can design reliable, observable AI systems.

Prerequisites

  • Python 3.x and a code editor or IDE installed on your computer
  • Course GitHub repository cloned - https://github.com/sujatas93/ai-native-llm-architecture-workshop
  • API credentials for an LLM provider (setup instructions provided in advance)
  • Experience building backend or API-based services
  • Working knowledge of Python or a similar programming language
  • Familiarity with basic LLM concepts (no deep ML background required)

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Why traditional patterns break (35 minutes)

  • Presentation: Deterministic versus probabilistic systems; where LLM apps fail in production systems
  • Hands-on exercise: Identify and categorize common LLM failure modes
  • Q&A

Input and behavior patterns (60 minutes)

  • Presentation: Prompt structure, context framing, contracts; how input design shapes model behavior and reliability
  • Hands-on exercises: Implement structured prompting, add prompt versioning, and run prompt experiments
  • Q&A
  • Break

Retrieval and knowledge patterns (30 minutes)

  • Presentation: Retrieval and knowledge patterns; when retrieval improves grounding versus when it degrades outcomes
  • Hands-on exercise: Add retrieval to an existing flow and evaluate grounding improvements
  • Q&A

Decisioning, fallbacks, and governance (50 minutes)

  • Presentation: Guardrails, escalation strategies (HITL), and safety boundaries; trade-offs between automation, safety, and user experience
  • Hands-on exercise: Implement fallback paths and policy checks for failure scenarios
  • Q&A
  • Break

Monitoring and observability (35 minutes)

  • Presentation: LLM observability (tracing, evaluation, and human feedback); selecting meaningful evaluation metrics for LLM systems
  • Hands-on exercise: Define evaluation signals and monitoring hooks for a sample app
  • Q&A

Closing the loop (30 minutes)

  • Presentation: End-to-end system thinking for LLM applications; how inputs, retrieval, decisioning, and observability reinforce each other
  • Hands-on exercise: Define evaluation signals and monitoring hooks for a sample app
  • Q&A

Your Instructors

  • Sujata Sridharan

    Sujata Sridharan is a Senior Software Engineer at Bolt Financial, where she builds AI-driven commerce infrastructure for large-scale, customer-facing systems. Her work spans applied AI, distributed backend systems, and reliability engineering, with prior experience at Microsoft and Amazon. She focuses on practical frameworks for building trustworthy, production-grade AI systems.

  • Vrinda Bhatia

    Vrinda Bhatia is a Senior Software Engineer at Block, where she works on large-scale infrastructure for ML inference and fraud prevention. She previously contributed to AWS AppStream and has experience building systems used by enterprise and public-sector organizations. Her work centers on reliability, scale, and production ML systems.

Skill covered

Software Architecture