Introduction to Generative AI

2 Training large language models

This chapter covers

• Explaining how LLMs are trained
• Introducing the emergent properties of LLMs
• Exploring the harms and vulnerabilities that come from training LLMs

For decades, the digital economy has run on the currency of data. The digital economy of collecting and trading information about who we are and what we do online is worth trillions of dollars, and as more of our daily activities have moved on to the internet, the mill has ever more grist to grind through. Large language models (LLMs) are inventions of the internet age, emulating human language by vacuuming up terabytes of text data found online.

The process has yielded both predictable and unpredictable results. Notably, there are significant questions ...