Python Natural Language Processing Cookbook - Second Edition

10

Generative AI and Large Language Models

In this chapter, we will explore recipes that use the generative aspect of the transformer models to generate text. As we touched upon the same in Chapter 8, Transformers and Their Applications, the generative aspect of the transformer models uses the decoder component of the transformer network. The decoder component is responsible for generating text based on the provided context.

With the advent of the General Purpose Transformers (GPT) family of Large Language Models (LLMs), these have only grown in size and capability with each new version. LLMs such as GPT-4 have been trained on large corpora of text and can match or beat their state-of-the-art counterparts in many NLP tasks. These LLMs have also ...