Generative Deep Learning, 2nd Edition

Part III. Applications

In Part III, we will explore some of the key applications of the generative modeling techniques that we have seen so far, across images, text, music, and games. We will also see how these domains can be traversed using state-of-the-art multimodal models.

In Chapter 9 we shall turn our attention to Transformers, a start-of-the-art architecture that powers most modern-day text generation models. In particular, we shall explore the inner workings of GPT and build our own version using Keras, and we’ll see how it forms the foundation of tools such as ChatGPT.

In Chapter 10 we will look at some of the most important GAN architectures that have influenced image generation, including ProGAN, StyleGAN, StyleGAN2, SAGAN, BigGAN, VQ-GAN, and ViT VQ-GAN. We shall explore the key contributions of each and look to understand how the technique has evolved over time.

Chapter 11 looks at music generation, which presents additional challenges such as modeling musical pitch and rhythm. We’ll see that many of the techniques that work for text generation (such as Transformers) can also be applied in this domain, but we’ll also explore a deep learning architecture known as MuseGAN that applies a GAN-based approach to generating music.

Chapter 12 shows how generative models can be used within other machine learning domains, such as reinforcement learning. We will focus on the “World Models” paper, which shows how a generative model can be used as the environment in which the ...