AI Codecon: Building with Open Source AI

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Introduction – Tim O’Reilly and Addy Osmani (10 minutes)

Tim and Addy welcome you to Building with Open Source AI.

Raffi Krikorian and Tim O’Reilly Fireside Chat (20 minutes)

Tim O’Reilly chats with Mozilla chief technology officer, Raffi Krikorian, about this critical moment for open source AI models and infrastructure. At Mozilla, Raffi has worked toward making open source AI as capable and accessible as proprietary alternatives, helping to ensure the future of this transformative technology is owned by the people who use it. Tim and Raffi will explore whether industry is shifting toward open weight models, candidly weigh the potential pros and cons, and unpack Raffi’s thesis on how practitioners can actively leverage these open tools today. Raffi argues that there’s a choice between owning versus renting our intelligence and that relying on closed, centralized platforms turns developers and users into tenants. He’ll outline a practical roadmap for an emerging integrated open stack that connects scattered open source tools. In this imagined future, diffusion wins and enables a more sovereign AI that matches the performance of proprietary platforms while allowing more control and flexibility.

AI Potluck: An Invitation to Build the Public AI Stack – Ayah Bdeir (20 minutes)

The open source AI ecosystem is full of incredible building blocks, from foundation models through inference backends and safety tooling. What’s been missing is the work of bringing them together into one product that no single company, country, or funder controls. Ayah Bdeir, CEO of Current AI, argues that the ecosystem needs an orchestrator and a collective roadmap. That’s the idea behind AI Potluck, an open project where each contributor brings the thing they do best. The first proof is Alpha, a free public AI assistant built in seven weeks with ten collaborators, entirely from existing open source components. Ayah will share how the assembly came together and take you through the Open Source AI Gap Map, a live view of what exists and what’s missing. Every gap on the map is an open invitation. An open AI stack is not just possible, it’s already being built.

Owning the Agentic Loop – Angie Jones (20 minutes)

AI agents make decisions and take actions on our behalf, which creates a new ownership challenge. Teams need to see what happened, control what an agent is allowed to do, and change the system as their needs evolve. But that’s difficult to do when so much of the stack is closed and proprietary. Angie Jones, VP of the Agentic AI Foundation, explores the agentic loop as the unit of ownership. She’ll look at the agentic runtime making decisions, the tools and data the agent can access, and the control plane that watches and constrains the whole flow. Along the way, you’ll hear how open source infrastructure gives teams the access they need to own each layer.

Break (5 minutes)

The Desktop Frontier – Ahmad M. Osman (20 minutes)

Ahmad M. Osman, founder and CEO of Osmantic, examines how rapidly improving capability density is moving advanced AI out of centralized data centers and onto hardware that individuals and organizations can own. Tracing the evolution of open models from Llama 2 through Qwen 2.5, DeepSeek R1, GLM 5.2, and Kimi K3, he introduces the concepts of “impact per parameter” and the Densing Law to explain how newer models are delivering greater intelligence with smaller active footprints. Capabilities that once required data-center-scale infrastructure can now run locally on devices as small as a smartphone. Ahmad makes the strategic case for open source self-hosted AI, arguing that owning your models, hardware, and sovereign compute stack is becoming an increasingly practical path toward greater control, privacy, cost predictability, resilience, and long-term value.

Local Off-Loading: Beyond Tokenmaxxing and Tokenminning – Gabe Goodhart (5 minutes)

Not all tokens are created equal. The same goes for tasks. Today’s debate between maximizing versus minimizing token usage misses this key nuance. Gabe Goodhart, chief architect for the AI Foundations division of IBM Research, shows how off-loading scoped tasks to local models can help to minimize AI costs while maximizing agent value, all without complex user orchestration logic.

Building an Adaptive LLM Router with Open Source – Isha Chaturvedi (5 minutes)

Not every prompt needs the same language model. Isha Chaturvedi, founding technical member at Keenable AI, shows you how to build a lightweight routing layer using Pydantic AI and OpenRouter. It automatically routes prompts to different LLMs based on task complexity. Isha compares routing decisions, latency, and inference cost, and discusses the trade-offs involved in building more efficient AI applications using open source tools.

Let the Workflow Decide: Routing Between Open and Closed Models – Sunny Israni (5 minutes)

Most AI workflows are built on whichever frontier model happens to be convenient. Sunny Israni, cofounder and CTO of Lightswitch, presents a lightning demo of a simple n8n workflow running on a large closed model, which he swaps out for a smaller open source model, showing how little you actually give up on a well-chosen task. The demo closes with a routing step that decides, task by task, whether work should go to an open or closed model based on sensitivity and cost.

Avoiding Prompt Debt: Let the Model Write the Prompts – Drew Breunig (20 minutes)

AI programs start the same way: with a plain English prompt and a working prototype by dinner. Then the bill arrives slowly, disguised as progress. Users flag edge cases, you add instructions, the instructions start contradicting each other, and six months later you’re staring at 2,000 tokens of escalating threats that nobody on the team will touch. Worse, your program is now locked to a specific model. This is prompt debt, and it’s why many teams are stuck on older models or unable to quickly evaluate new models and techniques. Drew Breunig, CEO and cofounder of cmpnd.ai, covers how to spot prompt debt, how to avoid it, and how to escape it. You’ll see when and how to set up systems that allow models to generate better prompts and hold these prompts accountable. You’ll also hear how to use prompt optimizers like GEPA and frameworks like DSPy to iterate quickly without breaking your systems and how to get started with evals in a simple and less intimidating way.

How Anaconda Built Its Model Curation Pipeline from Lessons Learned in Open Source Python – Seth Clark (Sponsored by Anaconda) (20 minutes)

Anaconda builds and maintains over 17,000 open source Python packages alongside a catalog of more than 70 open source AI models. Join Seth Clark, VP of product, to learn what trends Anaconda has seen among serving, quantization, fine-tuning, evals, and guardrails projects and how the company used these insights to build its own catalog of curated models. Seth’s talk includes the nuts and bolts of running, quantizing, benchmarking, and running red teaming on large open source models, as well as the major pitfalls. You’ll also hear which aspects of a model catalog are most important to get right at the enterprise level.

This session will be followed by a 30-minute Q&A in a breakout room. Stop by if you have questions for Seth.

Break (5 minutes)

Open Source AI Meets the State: What Builders Need to Know – Mike Linksvayer (20 minutes)

Governments increasingly see open models as strategic assets, security concerns, and targets of regulation. Sovereign AI initiatives may direct investment and procurement toward domestic ecosystems, while distrust of foreign models creates new headwinds. The EU AI Act and a growing patchwork of US state laws offer certain exemptions and accommodations for open source, but those protections don’t necessarily extend to the products built with it. For practitioners, the result is more complicated than “open means control.” Model provenance can affect customer trust and market access; regulatory obligations can change as a model moves from repository to product; and a politically favored ecosystem may create opportunities without providing a durable business advantage. Mike Linksvayer, VP of developer policy at GitHub, cuts through both policy hype and compliance anxiety, showing builders where geopolitical and regulatory developments may impact practical decisions, how to recognize when an open source carve-out ends, and when policy warrants attention or when it should take a back seat to shipping and finding product-market fit.

Lessons Learned When Fable Disappeared – Gene Kim (20 minutes)

What happens when the long-running AI agent that you depend on suddenly disappears? Gene Kim shares the story of the "Fable 5" disruption, what broke when the largest model went away, and the practical lessons it revealed about building resilient, AI-native engineering systems. The experience shows why you need a "Chaos Monkey for agents" to ensure smaller models can take over when needed, as well as better documentation, observability, and suspicion as AI agents increasingly become part of how work gets done.

BAML: How a Programming Language Can Help You Not Read the Code – Vaibhav Gupta (20 minutes)

Whether you like it or not, it’s no longer possible to read all the code that’s generated by a model. The solution cannot be, "Just ship it," or you end up with slop everywhere. Neither can the solution be "Read everything," because that’s impractical and doesn’t take advantage of one of the greatest inventions in history. Just as TypeScript enabled large-scale JavaScript applications to emerge, BAML - an open source, typed language and toolchain for defining, testing, and reliably integrating LLM prompts and structured outputs into applications - enables large-scale code bases that you can understand without reading every line. Vaibhav Gupta, founder and CEO of Boundary, shows you the language his company is developing in action.

Closing Remarks – Tim O’Reilly and Addy Osmani (5 minutes)

Tim and Addy close out today’s event.