AI Codecon: Software Craftsmanship in the Age of AI

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Introduction – Tim O’Reilly and Addy Osmani (10 minutes)

Tim and Addy welcome you to Software Craftsmanship in the Age of AI.

Cat Wu and Addy Osmani Fireside Chat (25 minutes)

We're witnessing a robust reshuffling of work, skills, and priorities in 2026. How is the role of the software engineer shifting to meet this moment? And how should engineers respond to these developments in real time? Cat Wu, product lead of Claude Code, and her team at Anthropic have some answers, drawn from their extensive integration of AI-assisted development into their workflows—as 90% of Anthropic’s code is now written by Claude. Join Cat and Addy for a chat about the superpowers Claude Code enables and the future of agentic coding.

How to Create a Team of Agents in OpenClaw and Ship Code with One Command – Ryan Carson (20 minutes)

The promise of AI agents often falls short due to "hallucinated state" and unreliable outputs. Ryan Carson, founder and CEO/CTO of a stealth startup, introduces Antfarm, an open source framework built on the OpenClaw ecosystem that transforms unpredictable LLMs into specialized, deterministic agent teams. You’ll discover how to move beyond single-prompt engineering by defining multi-agent workflows—Planner, Developer, Verifier, and Tester—using simple YAML and the Ralph loop pattern, then learn how to build a self-correcting pipeline where agents verify each other’s work in fresh, isolated contexts to ship production-ready code with a single command. Join in for insights on running complex workflows with zero infrastructure, implementing independent verification with verification gates, architecting repeatable playbooks to eliminate prompt drift, and more.

Orchestrating Coding Agents: Patterns for Coordinating Agents in Real-World Software Workflows – Addy Osmani (20 minutes)

The solo AI coding assistant era—one prompt, one response—is giving way to a multi-agent parallel execution model where developers act less like typists and more like conductors. Now engineers can spin up multiple agent instances that operate concurrently, each with a defined scope and isolated workspace. These agents can run in parallel, tackle distinct tasks, and communicate through structured channels without overwhelming individual context windows—effectively turning a single large task into composable, manageable subtasks with clear ownership. Addy Osmani unpacks the emerging orchestration patterns that are enabling developers to coordinate AI agents across roles, responsibilities, and phases of software work. Join him for actionable insights on how to make orchestration a first-class part of your engineering toolkit: how to decompose work for agents, manage interactions among them, and leverage orchestration patterns that scale with the complexity of modern codebases.

Break (5 minutes)

The Hidden Cost of Agentic Failure and the Next Phase of Agentic AI – Nicole Koenigstein (20 minutes)

Multi-agent systems rarely fail because of weak models. They fail because errors propagate. In production, agentic systems behave like probabilistic pipelines. Every unchecked handoff or tool call compounds risk, cost, and instability. This shows up as looping supervisors, runaway token usage, brittle workflows, spiking API costs, and failures that are hard to reproduce. AI researcher Nicole Koenigstein shows why improving individual agents does little for overall system reliability, and why the next phase of agentic AI is system level optimization. You’ll explore practical patterns for breaking failure loops using validation boundaries, LLMs as judges, structured collaboration, and reinforcement learning to teach agents how to collaborate more effectively with each other and use tools more efficiently. The focus of this session isn't on better prompts but on how to design, observe, and continuously optimize multi-agent systems so they improve instead of drifting or collapsing in production.

From Prompt to Multi-Agent System: The Evolution of Our AI Product – Hila Fox (25 minutes)

“Once we were an agent, then an agentic agent, and now a full-blown multi-agent system. When I say ‘once,’ I mean one year ago. Remember when you could build products that didn't change under your feet every 5 minutes? I do. It was nice,” quips Qodo principal engineer Hila Fox. Hila shares the company’s journey through AI's chaos: “agentifying” Qodo’s code generation agent, architecting a multi-agent code review system using LangGraph and MCPs, and preparing for the inevitable next shift—because in AI, the only constant is change.

When AI Agents Break Production and What We Did About It – Advait Patel (5 minutes)

Broadcom began using background AI agents in its production cloud environment to analyze logs, propose fixes, and refactor parts of the codebase asynchronously. One of those agents generated a change that looked correct, passed tests, and merged cleanly but caused subtle production degradation hours later. This talk is a short, true war story about that failure. Join senior site reliability engineer Advait Patel for insights on how to adapt software craftsmanship to a world where parts of our system reason, plan, and act on their own.

Is Your Eval Lying to You? Catching Hidden Failures in Agent Evaluation – Abhimanyu Anand (5 minutes)

If you're evaluating AI agents, you've likely encountered three common failures. The first is the hidden trade-off: Improving accuracy with a tool change also increases tool calls and latency, and a single positive metric masks overall degradation. The second is the sycophantic judge: LLM-based evaluators are nondeterministic, so a score increase may only reflect sensitivity to a prompt change, not an improved user experience. Finally, the noise floor means that without significance testing, you’re often shipping coin-flip gains that will disappear on the next run. Senior data scientist Abhimanyu Anand takes you through how Elastic addressed these failures, using a real experiment as an example. You’ll explore the evaluation setup, significance testing, prompt structure behind the evaluator, and more—and leave with a clear design for auditing your own agents.

Break (5 minutes)

The Mythical Agent-Month – Wes McKinney (20 minutes)

“I run parallel Claude Code sessions all day, write code in languages I've never written by hand, and ship faster than ever. But I'm also watching my agents chase their own tails in the bloated codebases they've created,” says Wes McKinney. Fifty years ago, Fred Brooks argued that adding programmers to a late project makes it later—and his deeper lesson is proving uncomfortably durable in the agentic era. Agents are extraordinary at eliminating accidental complexity, but they can't tell accidental from essential, so they often generate new complexity at machine speed. Join Wes to discover why, when code is free, knowing what to build—and not to build—is the last skill that matters.

From Automation to Augmentation: Designing AI Coaches That Amplify Expertise – Mike Amundsen (10 minutes)

Generative AI excels at producing answers, but producing one-off answers isn’t the same thing as developing durable capability. Recent research from Anthropic suggests that heavy reliance on generative systems can improve short-term performance while weakening long-term skill formation. That realization isn’t new. Douglas Engelbart argued that computers should augment human intellect, not displace it. J. C. R. Licklider imagined a partnership in reasoning, not a delegation of thought. In this talk, Mike Amundsen explores a practical alternative. By separating work into distinct cognitive phases, expanding possibilities, structuring and interrogating ideas, and executing deliberately, we can design interactive AI experiences that strengthen human reasoning instead of bypassing it. AI coaches are one concrete implementation of this approach. Join Mike for an introduction to the skill formation challenge, examine a better pattern for working with AI, and get a live demo of an AI coach in action.

Agent-to-Agent Collaboration and Agent Provenance – Tatiana Botskina (5 minutes)

As AI agents move into production, the main challenges shift from model quality to coordination, trust, and traceability. Agent-to-agent collaboration breaks without clear agent provenance: Without ownership, versioning, lineage, and trust signals, multi-agent systems become opaque, fragile, and unsafe. AI researcher Tatiana Botskina introduces agent registries as a missing infrastructure layer that enables discoverable, composable, and auditable agents, and outlines practical design patterns for building collaborative agent systems that can be operated and governed in production.

Automating MCP Server Testing: Engineering Reliability for Agentic Systems – Neethu Elizabeth Simon (5 minutes)

When Arm began deploying a custom MCP server to power AI assistants with tools and knowledge for architecture development, migration, and optimization, the team quickly ran into a complication: How do we test a system designed for nondeterministic interaction? Arm staff solution architect Neethu Elizabeth Simon explains how they built a repeatable CI-enforced testing strategy for Arm’s custom MCP server using pytest and Testcontainers, moving from manual validation to fully automated containerized integration tests that spin up real MCP servers in Docker on every pull request. This isn’t about mocking LLM outputs. It’s about testing the contract between agents and tools and treating the MCP server like production infrastructure, not experimental glue code.

How Multi-Agent Systems Enable Personalized Messaging at Scale – Arushee Garg (5 minutes)

Writing a strong job application message can make the difference between being noticed or ignored by a recruiter. LinkedIn built an AI system that generates personalized jobseeker messages to help members stand out when applying to roles, but what looks like a simple message generator is actually a coordinated production system. LinkedIn senior software engineer Arushee Garg explains how the company’s Premium team built a supervisor-based multi-agent workflow using LangGraph that analyzes user context, job signals, and persona to generate tailored recruiter-facing messages. You’ll leave with a practical understanding of how personalized messages can be generated at scale, along with lessons on prompt management, evaluation, and operating agentic systems in production.

Break (5 minutes)

The AI Flower: A Public Capability Architecture for AI-Native Engineering – Juliette van der Laarse (20 minutes)

The current AI landscape is saturated with tools, courses, and playbooks, and the bottleneck has shifted from information to orientation and integration: knowing what matters, where to start, and how to embed AI into engineering work in a way that holds beyond the hype. Juliette van der Laarse, engineering manager at an international financial services company, takes you through the AI Flower, a capability architecture designed to structure that overload and guide engineers and teams from scattered experimentation to reliable integration. You’ll learn how the model is structured, how the proposed skill fossilization lens keeps it stable as tools and techniques shift, and how to use the AI Flower as a practical navigation aid to map current practice, identify capability gaps, and choose sensible next steps—without lowering the bar on quality and accountability.

Aaron Levie and Tim O’Reilly Fireside Chat (20 minutes)

Aaron Levie is cofounder and CEO of Box, where he spearheads the evolution of the company’s Intelligent Content Management platform to meet the needs of the agentic era. According to Aaron, we’re living in “an era of context for AI”—success for enterprises will be measured by how well context is managed and how well enterprises adjust their workflows to enable agents to work effectively. File systems, he posits, are a “core primitive” for agentic systems: “Agents are insanely good at working with file systems as they offer the perfect environment for manipulation of data and managing context for what agents need to work on.” Aaron joins Tim O’Reilly to discuss the future of building software in the agentic era; the challenges of managing context and imagining new agentic identities; the importance of building API-first software, and more.

Closing Remarks – Tim O’Reilly and Addy Osmani (5 minutes)

Tim and Addy close out today’s event.