February 2024
33m
English
audiobook
Generative AI in the Real World: Pete Warden on Running AI on Small Systems
by Ben Lorica, Pete Warden
Overview
Pete Warden, founder of Useful Sensors and co-author of TinyML, discusses use cases for artificial intelligence that we rarely think about: how can you run AI on very small systems? How can you put AI on consumer devices in ways that are actually useful and not just buzzword-compliant? AI doesn’t have to rely on massive GPU farms. Pete talks about what happens when you exchange one set of requirements (extreme power, heat, and expense) for another (minimal size, cost, and heat).
About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2024, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.
Points of Interest
- 00:00: Introductions, including Pete’s introduction to his company.
- 2:22: What are some of the challenges and use cases for sensor-driven AI?
- 4:11: Is sensor-driven AI relevant to industries other than hardware?
- 6:22: Now we’re in the age of foundation models and large language models. Is “large” incompatible with “tiny”? Can you run language models on smaller devices?
- 8:00: Will there be developments in tinyML that will benefit the broader LLM community?
- 9:30: What’s deployable today in computer vision, speech, and language? What can be done with hardware that’s constrained by cost, size, and power consumption?
- 11:15: How will product designers work with sensor-driven AI? Will they simply select from a palette of optional modules?
- 12:37: Pete walks us through the development of AI-in-a-Box, from its conception to its reception.
- 15:31: Your devices don’t have network connections. Without a network connection, how do you update models? Is it necessary?
- 19:00: Do you do Retrieval Augmented Generation (RAG) on your devices?
- 20:35: Our devices have user interfaces that combine voice and presence. A voice interface is central, but visual (and other channels) help to create an awareness of the speaker.
- 21:35: What are some of your specific challenges, like power consumption and latency? How do you make tradeoffs?
- 22:45: What is the future of large language models for sensor-driven AI?
- 26:50: What are some of the security concerns for sensor-driven AI and what are you doing about them?
- 28:22: What is Dark Compute and why is it important?
- 30:48: What are the biggest opportunities for pushing AI into consumer devices? We need to start with problems that users actually care about.
- 32:30: How can listeners connect to the broader movement around TinyML?
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Listen now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.