Generative Deep Learning, 2nd Edition

Chapter 10. Advanced GANs

Chapter Goals

In this chapter you will:

• See how a ProGAN model progressively trains a GAN to generate high-resolution images.

• Understand how ProGAN was adapted to build StyleGAN, a high-performing GAN for image synthesis.

• Explore how StyleGAN was adjusted to create StyleGAN2, a state-of-the-art model that improves further upon the original work.

• Learn about the key contributions of these models, including progressive training, adaptive instance normalization, weight modulation and demodulation, and path length regularization.

• Walk through the architecture of the Self-Attention GAN (SAGAN), which incorporates the attention mechanism into the GAN framework.

• See how BigGAN expands upon the ideas in the SAGAN paper to produce high-quality images.

• Learn how VQ-GAN uses a codebook to encode images into a discrete sequence of tokens that can be modeled using a Transformer.

• See how ViT VQ-GAN adapts the VQ-GAN architecture to use Transformers instead of convolutional layers in the encoder and decoder.

Chapter 4 introduced generative adversarial networks (GANs), a class of generative model that has produced state-of-the-art results across a wide variety of image generation tasks. The flexibility in the model architecture and training process has led academics and deep learning practitioners to find new ways to design and train GANs, leading to many different advanced flavors of the architecture that we shall explore in this chapter.

Introduction ...