Generative Deep Learning, 2nd Edition

Chapter 8. Diffusion Models

Chapter Goals

In this chapter you will:

• Learn the underlying principles and components that define a diffusion model.

• See how the forward process is used to add noise to the training set of images.

• Understand the reparameterization trick and why it is important.

• Explore different forms of forward diffusion scheduling.

• Understand the reverse diffusion process and how it relates to the forward noising process.

• Explore the architecture of the U-Net, which is used to parameterize the reverse diffusion process.

• Build your own denoising diffusion model (DDM) using Keras to generate images of flowers.

• Sample new images of flowers from your model.

• Explore the effect of the number of diffusion steps on image quality and interpolate between two images in the latent space.

Alongside GANs, diffusion models are one of the most influential and impactful generative modeling techniques for image generation to have been introduced over the last decade. Across many benchmarks, diffusion models now outperform previously state-of-the-art GANs and are quickly becoming the go-to choice for generative modeling practitioners, particularly for visual domains (e.g., OpenAI’s DALL.E 2 and Google’s ImageGen for text-to-image generation). Recently, there has been an explosion of diffusion models being applied across wide range of tasks, reminiscent of the GAN proliferation that took place between 2017–2020.

Many of the core ideas that underpin diffusion models ...