Building AI Agents with LLMs
Published by O'Reilly Media, Inc.
Harnessing the power of generative AI with autonomous agents
The development and release of ChatGPT and other state-of-the-art large language models (LLMs) has renewed interest in the concept of AI agents, a term that evokes a range of assumptions about their potential as applications to independently automate many tasks along with the risks they present if not constrained. In simple terms, an AI agent is an AI entity that is given a description of a task, autonomously retrieves the tools it needs to execute the task, and then executes it.
In this event, our third in an ongoing series of special events on ChatGPT, LLMs, and generative AI, we’ll discuss the different types of AI agents and how the latest large AI models are making these AI agents more powerful than ever. Integrating these AI agents into your products and services presents some challenges that aren’t usually of concern for web-based ML models. These include how to design a good interface that can take user inputs, show the inferences and communicate the progress, logging costs through each step and monitoring the performance while understanding user preferences, editing plans, etc. Our experts will present some of the tools you need to create your own AI agents and show you how they can help your organization achieve higher levels of efficiency and productivity by making AI your assistant.
What you’ll learn and how you can apply it
- Explore the different types of AI agents and when and how to use them
- Understand which agents to use based on your particular use case—for example, reasoning-based agents versus sufficiency agents versus delegation agents
- Understand the limitations of AI agents and how to contain the scope of their work
- Learn about the latest tools that have been developed to help you build your own AI agent
This live event is for you because...
- You’re curious about AI agents and how they can help your organization.
- You’re interested in knowing how others are using AI agents and the tools that are being developed to enable developers to create and use them.
- You’re interested in exploring whether to build your own agents or use proprietary services like Cognosys.
Prerequisites
- Come with your questions
- Have a pen and paper handy to capture notes, insights, and inspiration
Recommended follow-up:
- Read Designing Autonomous AI (book)
- Read Generative Deep Learning, second edition (book)
Schedule
The time frames are only estimates and may vary according to how the class is progressing.
Abi Aryan and Nicole Butterfield: Introduction (5 minutes) - 9:00am PT | 12:00pm ET | 4:00pm UTC/GMT
- Abi and Nicole welcome you to the event.
Kence Anderson: Intelligent Autonomous Agents (20 minutes) - 9:05am PT | 12:05pm ET | 4:05pm UTC/GMT
- Autonomous driving and AutoGPT have captured our imagination about automated AI systems that make real-time decisions. While intelligent automation has been around for a long time, modern artificial intelligence promises more human-like decision-making as well as success in more diverse situations. Kence Anderson, CEO of Composabl and author of Designing Autonomous AI, discusses and demonstrates how to build intelligent autonomous agents that can succeed in real-world applications.
- Kence Anderson teaches autonomous AI design and consults for enterprises on how to build their autonomous systems organizations and practices. Previously, he served as director of autonomous AI adoption at Microsoft. Kence has pioneered uses for autonomous AI in industry and designed over 150 autonomous decision-making AI systems for large enterprises.
Yohei Nakajima: Building an Autonomous Agent—The Evolution of BabyAGI (20 minutes) - 9:25am PT | 12:25pm ET | 4:25pm UTC/GMT
- Yohei Nakajima, creator of BabyAGI, breaks down the functionality of code that makes up an autonomous agent by sharing the evolution of BabyAGI and its modifications. He’ll outline various approaches to LLM-based task creation, discuss skill and tool management, and demonstrate the most recent autonomous agents.
- Yohei Nakajima is a venture capitalist by day and a builder by night. He’s general partner at Untapped Capital, a pre-seed VC firm investing in unexpected founders. He approaches VC through a unique lens: building in public with the latest technologies, both to learn and to meet founders. Leveraging AI across all of his work, he built Untapped Capital’s custom no-code operating system, launched an NFT collection called PixelBeasts, and built more than 50 prototypes—most recently BabyAGI—for apps built on top of LLMs.
Jenny Kaehms: Building Your First AI Agents (20 minutes) - 9:45am PT | 12:45pm ET | 4:45pm UTC/GMT
- Jenny Kaehms discusses early agent use cases and gives an overview of the AI agent market before diving into personal and enterprise agent use cases. This is a unique opportunity for founders or project leaders to understand the potential use cases for agents and how to scope agent projects, starting with more basic implementations and moving to advanced scenarios.
- Jennifer Kaehms is an engineer-turned-venture-capitalist. She's a partner at Foundation Capital, a seed VC firm investing in AI. She’s been investing in the AI space since 2016, working with companies including Lambda Labs, Luminar, and Forethought.
- Break (10 minutes)
Div Garg: Building Human-Like AI Agents—Solving Challenges Around Computer Interaction, Long-Term Memory, and Agent-Agent Communication (20 minutes) - 10:15am PT | 1:15pm ET | 5:15pm UTC/GMT
- Autonomous AI agents offer unprecedented potential in improving everyday productivity and revolutionizing how we interact with software. However, most current AI agents present unique challenges such as a lack of reliability and drift from the original goal. These hurdles become more pronounced when developing production-ready agents. Div Garg shows you how to tackle these challenges to build scalable and useful AI agents by using a human-inspired perspective. He also discusses some frameworks for building generalized AI systems and adding actions, long-term memory, and agent-agent communication.
- Div Garg is the cofounder and CEO of MultiOn AI, a company focused on developing human-like AI agents that can interact with computers and assist with everyday tasks. Div has spent his career at the intersection of ML, research, and startups; most recently, he led AI at Collaborative Robotics. He’s enthralled with building intelligent systems and has worked on various high-impact AI projects including self-driving cars, robotic control, and Minecraft AI agents. He also created and taught a course on transformers at Stanford University, where he was a PhD student.
Shawn “Swyx'' Wang: Anatomy of Autonomy (20 minutes) - 10:35am PT | 1:35pm ET | 5:35pm UTC/GMT
- The fifth killer app of AI is here, and it is the autonomous agent. Swyx takes you through the evolutionary building blocks that led to the state of agents today and discusses why the future of software 3.0 is a neat sum of 1.0 and 2.0—blending code and LLMs.
- Swyx is the founder of smol.ai and writer and cohost of the Latent.Space podcast.
Arjun Bansal: Evolution of LLM Agents and the Need for Robust Evaluations (20 minutes) - 10:55am PT | 1:55pm ET | 5:55pm UTC/GMT
- Arjun Bansal explores the evolution of large language model (LLM) agents and the growing need for robust evaluations. He traces the development of LLM agents from single API calls to dynamic chains that enable them to perform more complex tasks, and he discusses the potential productivity gains and the challenges that lie ahead. The importance of evaluations in improving the quality of LLM agents and enabling features such as fine-tuning, distillation, and model orchestration, will be highlighted.
- Arjun Bansal is an entrepreneur and AI expert focused on understanding and building intelligent systems. He is cofounder and CEO of Log10.io, a platform for building and deploying LLM-powered agents faster and more reliably. Previously, he cofounded XOKind (AI travel agents) and Nervana Systems (acquired by Intel). Arjun's career spans research in brain-machine interfaces, building AI processors and AI sidekicks.
Abi Aryan and Nicole Butterfield: Closing Remarks (5 minutes) - 11:15am PT | 2:15pm ET | 6:15pm UTC/GMT
- Abi and Nicole close out today’s event.
Your Hosts and Selected Speakers
Nicole Butterfield
Nicole Butterfield is senior acquisitions editor of AI content at O’Reilly. She has been working in technical publishing for 12+ years and has acquired content covering topics from programming and software development to data science and machine learning.
Abi Aryan
Abi Aryan is the founder of Abide AI and a machine learning engineer with over nine years of experience developing production-level ML systems. She’s also the author of LLMOps: Managing Large Language Models in Production (O’Reilly) and research papers in AutoML, multi-agent systems, and LLMs. Previously, she was a visiting research scholar at the Cognitive Sciences Lab at UCLA, and serves as a reviewer for the ACL, EMNLP, and AABI conferences.