Infrastructure & Ops Superstream: Infrastructure for AI

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Sam Newman: Introduction (5 minutes)

Sam Newman welcomes you to the Infrastructure & Ops Superstream.

An Introduction to AI Infrastructure – Bryan Oliver (35 minutes)

AI has drastically changed computing infrastructure, forcing large-scale innovation in the GPU computing sector. We no longer train or run inference on just one GPU; instead, we use thousands of GPUs simultaneously for single models. This type of infrastructure is very complex, and we must think differently about how we write and deploy software to it. Factors such as data center topology and even rack-to-rack latency are now more important than ever. Bryan Oliver, author and Thoughtworks engineer, gives you an overview of AI infrastructure along with entry-level breakdowns of its inner workings. You’ll walk away with a greater respect for the scale and complexity of this exploding technology and a general understanding of how you can start to think about using it.

Infrastructure for AI: Managing GPU Workloads with Kubernetes – Saiyam Pathak (35 minutes)

The AI revolution has made GPUs the most valuable resource in modern infrastructure and the hardest to manage at scale. Kubernetes is becoming the platform of choice for AI workloads, but production-grade GenAI comes with real challenges: GPU scarcity, poor utilization, multiteam access, and fragmented deployments across cloud and bare metal. Saiyam Pathak, head of developer relations at vCluster, explores how to build AI-ready Kubernetes platforms that efficiently run today’s LLM-powered workloads. The session covers GPU sharing (time-slicing, MIG), advanced scheduling with Kai Scheduler, and secure multitenancy using vCluster. You’ll also learn how open source tools like vLLM fit into real architectures for scalable inference. You’ll come away with a practical blueprint for delivering high-performance, cost-efficient, multitenant AI infrastructure across Kubernetes environments.

Break (5 minutes)

Distributed Data Architectures for AI: A Deep Dive into Real-World Use Cases – Rob Reid (Sponsored by Cockroach Labs) (30 minutes)

Modern AI use cases such as recommendations, semantic search, RAG, agentic AI, and unified queries place heavy demands on the underlying database. Author and Cockroach Labs’ technical evangelist Rob Reid explores what each use case needs in terms of consistency, performance, scale, and resilience—and what happens when traditional systems are pushed beyond their limits. You’ll learn how distributed SQL architecture provides the most practical path to support AI at enterprise scale, with the ability to eliminate the complex data pipelines that emerge when data is spread across multiple systems. This session will be followed by a 30-minute Q&A in a breakout room. Stop by if you have more questions for Rob.

Protecting and Connecting High-Value AI Workloads on Kubernetes with Cilium – Nico Vibert (35 minutes)

ML models are precious assets and a central part of an organization’s intellectual capital, yet they now face a growing range of attacks, including prompt injection, model poisoning, and data exfiltration through misconfigured services. At the same time, AI and ML workloads demand high bandwidth, low latency, and predictable connectivity, and most modern AI platforms run on Kubernetes to meet these scaling needs. Nico Vibert, technical marketing engineering director at Isovalent and coauthor of Cilium: Up and Running, shows how Cilium’s eBPF-powered networking and security stack protects model serving pipelines while delivering the performance AI workloads require. You’ll learn how to secure interservice communication, enforce workload-aware policy, observe inference traffic, and build reliable high-throughput clusters for sensitive ML applications. This session gives platform teams the tools to safeguard their models and keep AI systems resilient at scale.

Break (5 minutes)

Building Multicloud AI Platforms Without the Pain – Romil Bhardwaj (35 minutes)

GenAI workloads are redefining how AI platforms are built. Teams can no longer rely on a single cloud to satisfy their GPU needs, infra costs are growing, and the productivity of ML engineers is paramount. Going multicloud secures GPU capacity, reduces costs, and eliminates vendor lock-in, but it also introduces operational complexity that can slow down ML teams. Romil Bhardwaj, cocreator of the SkyPilot open source project, provides a guide to building a multicloud AI platform that unifies cloud VMs and Kubernetes clusters across hyperscalers (AWS, GCP, and Azure), neoclouds (CoreWeave, Nebius, Lambda), and on-premises clusters into a single compute abstraction. You’ll learn practical implementation details including workload scheduling strategies, automated cloud selection, and dependency management. This approach lets ML engineers use the same interface for both interactive development sessions and large-scale distributed training jobs, enabling them to focus on building great AI products.

Building AI-Ready Infrastructure: Storage, Performance, and Resilience at Scale – Murat Karslioglu (35 minutes)

As AI transforms enterprise operations, infrastructure leaders must evolve their data foundations to meet unprecedented demands for scale, performance, and resilience. MinIO’s head of global field architects, Murat Karslioglu, explores how modern organizations are building AI-ready infrastructure through cloud native storage architectures, automation, and hybrid-cloud agility. Drawing on real-world use cases, he outlines strategies for unifying data across environments, optimizing performance and cost, and embedding resilience into core systems. You’ll leave with a practical framework for enabling scalable, efficient, and future-proof infrastructure, purpose-built for the AI era.

Sam Newman: Closing Remarks (5 minutes)

Sam Newman closes out today’s event.