How LLMs Understand & Generate Human Language

1+ Hours of Video Instruction

Your introduction to how generative large language models work.

Overview

Generative language models, such as ChatGPT and Microsoft Bing, are becoming a daily tool for a lot of us, but these models remain black boxes to many. How does ChatGPT know which word to output next? How does it understand the meaning of the text you prompt it with? Everyone, from those who have never once interacted with a chatbot, to those who do so regularly, can benefit from a basic understanding of how these language models function. This course answers some of your fundamental questions about how generative AI works.

In this course, you learn about word embeddings: not only how they are used in these models, but also how they can be leveraged to parse large amounts of textual information utilizing concepts such as vector storage and retrieval augmented generation. It is important to understand how these models work, so you know both what they are capable of and where their limitations lie.

About the Instructor

Kate Harwood is part of the Research and Development team at the New York Times, researching the integration of state-of-the-art large language models into the Times’ reporting and products. She also teaches introduction to AI courses through The Coding School. She has a MS in computer science from Columbia University. Her primary focus is on natural language processing and ethical AI.

Learn How To

• Understand how human language is translated into the math that models understand
• Understand how generative language models choose what words to output
• Understand why some prompting strategies and tasks with LLMs work better than others
• Understand what word embeddings are and how they are used to power LLMs
• Understand what vector storage/retrieval augmented generation is and why it is important
• Critically examine the results you get from large language models

Who Should Take This Course

Anyone who

• Is interested in demystifying generative language models
• Wants to be able to talk about these models with peers in an informed way
• Wants to unveil some of the mystery inside LLMs’ black boxes but does not have the time to dive deep into hands-on learning
• Has a potential use case for ChatGPT or other text-based generative AI or embedding storage methods in their work

Course Requirements

No specific requirements

Lesson Descriptions

Lesson 1: Introduction to LLMs and Generative AI

Lesson 1 is an introduction to large language models and generative artificial intelligence. Kate discusses what an LLM is and what generative AI is, and provides a general introduction to machine learning.

Lesson 2: Word Embeddings

Lesson 2 introduces word embeddings. Kate introduces the word embedding space and discusses how word embeddings capture word meanings, enabling LLMs to read and produce textual content. The lesson then turns to another AI concept, tokenization, followed by a discussion that pulls it all together. Kate finishes the lesson with an interesting side-effect of word embeddings.

Lesson 3: Word Embeddings in Generative Language Models

Lesson 3 begins with a discussion of how word embeddings are used in generative language models. Kate then introduces model architectures that use word embeddings, specifically recurrent neural networks (RNNs) and transformers. Kate covers the attention mechanism in transformers, contextual word embeddings, and how transformers are used for language generation. The lesson finishes with a discussion of what works well and what can go wrong when we train models on word embeddings.

Lesson 4: Other Use Cases for Embeddings

Lesson 4 covers how embeddings can also be used for summarization and vector storage. It finishes with an example of how embeddings can be used for retrieval augmented generation (RAG).

About Pearson Video Training

Pearson publishes expert-led video tutorials covering a wide selection of technology topics designed to teach you the skills you need to succeed. These professional and personal technology videos feature world-leading author instructors published by your trusted technology brands: Addison-Wesley, Cisco Press, Pearson IT Certification, Prentice Hall, Sams, and Que Topics include IT Certification, Network Security, Cisco Technology, Programming, Web Development, Mobile Development, and more. Learn more about Pearson Video training at http://www.informit.com/video.