Engineering AI Systems: Architecture and DevOps Essentials
by Len Bass, Qinghua Lu, Ingo Weber, Liming Zhu
4
Foundation Models
For data guzzling AI companies, the internet is too small.
—Wall Street Journal
IN CHAPTER 1, Introduction, we differentiated between traditional machine learning (ML) models, which are trained for narrow tasks, and foundation models (FMs), which are trained for general purposes through the synthesis of massive amounts of data such as natural language, code, images, and videos. In Chapter 3, AI Background, we explored ML models in more detail. In this chapter, we go into detail on FMs.
We discuss FMs in general, the transformer architecture that underlies their strong advances as well as some alternatives, and how you customize FMs. Then we explore the design considerations when you are building a system using FMs, the maturity ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access