AI Superstream: Retrieval-Augmented Generation (RAG) in Production

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Introduction – Susan Shu Chang (5 minutes)

• Susan Shu Chang welcomes you to the AI Superstream.

Keynote: Systems Over Models—Tackling High-Stakes Use Cases with RAG 2.0 – Douwe Kiela (15 minutes)

• RAG 2.0 is Contextual AI’s proprietary approach for solving mission-critical AI use cases, where accuracy requirements are high and there’s a low tolerance for mistakes and hallucinations. Douwe Kiela, Contextual AI’s cofounder and CEO, sheds light on how this approach differs from classic RAG, the common pitfalls and limitations while moving into production, and why AI practitioners would benefit from focusing less on individual model components and more on the systems-level perspective.

Beyond Traditional RAG: Dual-System Architecture and Efficient Retrieval – Faye Zhang (30 minutes)

• Transform your applications with recent advances in RAG that achieve 40% higher efficiency while handling 80,000-token contexts at a fraction of the computational cost. Pinterest staff software engineer Faye Zhang presents two practical approaches: a dual-system architecture that intelligently combines lightweight and powerful models, and a new retrieval technique that reduces GPU demands through dynamic sparsity. Learn how to implement these proven methods to build faster, more efficient, and more capable AI systems that excel at both simple queries and complex analytical tasks.

Break (5 minutes)

Building a Recommender System with RAG for LLMs – Apoorva Joshi (30 minutes)

• Before we started using generative AI models for everything, we were mostly building specialized models, trained on specific datasets, to solve specific problems. If we are to now use the same model for different tasks, we need to present the model with the most relevant data for the task at hand. Some might call this retrieval-augmented generation, but really it’s a good old recommender system for large language models instead of humans. Apoorva Joshi, an AI developer advocate at MongoDB, dives deep into hybrid search, a commonly used retrieval technique in RAG systems, and explains how combining it with metadata filtering and reranking algorithms results in a scalable recommender system for LLMs (a.k.a. RAG).

Authorizing LLM Responses by Filtering Vector Embeddings – Greg Sarjeant (Sponsored by Oso) (30 minutes)

• Large language models open up a new way to interact with your data: Your users can now use a natural language interface rather than traditional search to get back relevant information. But the flexibility and scale of LLMs make it harder to ensure that you don’t leak sensitive data. Oso developer advocate Greg Sarjeant explores these challenges and shows you how to use retrieval-augmented generation to build an authorized LLM chatbot that protects your data.
• This session will be followed by a 30-minute Q&A in a breakout room. Stop by if you have more questions for Greg.

Break (5 minutes)

Building Agentic RAG Applications in Minutes – Ofer Mendelevitch (30 minutes)

• Agentic RAG extends standard RAG applications into powerful AI assistants and agents with advanced capabilities. By equipping LLMs with access to external tools—such as real-time data sources, APIs, or database access—developers can elevate AI systems beyond simple query engines, turning them into powerful problem-solvers that adapt to dynamic needs. Ofer Mendelevitch, who leads developer relations at Vectara, takes you though building agentic RAG applications step-by-step, delving into the technical details of a real-world example and offering solutions to the challenges you’re likely to face, such as picking the right tools and preventing hallucinations.

Graph RAG: Building Production-Ready Agentic Systems – Anthony Alcaraz (30 minutes)

• Learn how to overcome the limitations of traditional AI approaches by leveraging graph-native architectures for enhanced reasoning capabilities and enterprise scalability. Anthony Alcaraz, AWS's senior AI/ML strategist for EMEA startups, discusses how to build intelligent systems using Graph RAG, integrating knowledge graphs, autonomous agents, and neural-symbolic reasoning. You’ll dive into practical implementation patterns, architectural principles, and real-world case studies that demonstrate how to design, develop, and deploy production-ready agentic systems.

Break (5 minutes)

Where Is RAG Going? Increased Indexing Time Compute for Better, Faster, Cheaper Answers – Max Buckley (30 minutes)

• The constraints and design decisions underpinning RAG came from limitations in the LLM models of an earlier era. As these constraints dissolve, we can seize the opportunity to redesign our systems to be simultaneously more powerful and more flexible. Max Buckley, a senior software engineer at Google, will examine common failure modes of traditional RAG systems and demonstrate how increasing indexing time compute can address these challenges, giving your RAG system access to enriched, more accurate chunks with more information, including precomputed answers for anticipated queries, enabling faster, more efficient responses.

Making RAG Pipelines Work in the Real World: Challenges and Solutions – Nikhil Pentapalli and Surabhi Bhargava (30 minutes)

• Adobe senior machine learning engineer Nikhil Pentapalli and machine learning tech lead Surabhi Bhargava outline real-world challenges and practical solutions for deploying retrieval-augmented generation systems at production scale. They’ll introduce the RAGOps stack, detailing the journey from taking a concept all the way to production while addressing key issues such as data and retrieval inefficiencies, latency management, cost optimization, evaluation, and more.

Closing Remarks – Susan Shu Chang (5 minutes)

• Susan Shu Chang closes out today’s event.