book

Generative Deep Learning, 2nd Edition

by David Foster

April 2023

Intermediate to advanced

456 pages

11h 12m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

Objective and ApproachPrerequisitesRoadmapChanges in the Second EditionOther ResourcesConventions Used in This BookCodebaseUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Generative Modeling?Generative Versus Discriminative ModelingThe Rise of Generative ModelingGenerative Modeling and AIOur First Generative ModelHello World!The Generative Modeling FrameworkRepresentation LearningCore Probability TheoryGenerative Model TaxonomyThe Generative Deep Learning CodebaseCloning the RepositoryUsing DockerRunning on a GPUSummary
Data for Deep LearningDeep Neural NetworksWhat Is a Neural Network?Learning High-Level FeaturesTensorFlow and KerasMultilayer Perceptron (MLP)Preparing the DataBuilding the ModelCompiling the ModelTraining the ModelEvaluating the ModelConvolutional Neural Network (CNN)Convolutional LayersBatch NormalizationDropoutBuilding the CNNTraining and Evaluating the CNNSummary
IntroductionAutoencodersThe Fashion-MNIST DatasetThe Autoencoder ArchitectureThe EncoderThe DecoderJoining the Encoder to the DecoderReconstructing ImagesVisualizing the Latent SpaceGenerating New ImagesVariational AutoencodersThe EncoderThe Loss FunctionTraining the Variational AutoencoderAnalysis of the Variational AutoencoderExploring the Latent SpaceThe CelebA DatasetTraining the Variational AutoencoderAnalysis of the Variational AutoencoderGenerating New FacesLatent Space ArithmeticMorphing Between FacesSummary
IntroductionDeep Convolutional GAN (DCGAN)The Bricks DatasetThe DiscriminatorThe GeneratorTraining the DCGANAnalysis of the DCGANGAN Training: Tips and TricksWasserstein GAN with Gradient Penalty (WGAN-GP)Wasserstein LossThe Lipschitz ConstraintEnforcing the Lipschitz ConstraintThe Gradient Penalty LossTraining the WGAN-GPAnalysis of the WGAN-GPConditional GAN (CGAN)CGAN ArchitectureTraining the CGANAnalysis of the CGANSummary
IntroductionLong Short-Term Memory Network (LSTM)The Recipes DatasetWorking with Text DataTokenizationCreating the Training SetThe LSTM ArchitectureThe Embedding LayerThe LSTM LayerThe LSTM CellTraining the LSTMAnalysis of the LSTMRecurrent Neural Network (RNN) ExtensionsStacked Recurrent NetworksGated Recurrent UnitsBidirectional CellsPixelCNNMasked Convolutional LayersResidual BlocksTraining the PixelCNNAnalysis of the PixelCNNMixture DistributionsSummary
IntroductionNormalizing FlowsChange of VariablesThe Jacobian DeterminantThe Change of Variables EquationRealNVPThe Two Moons DatasetCoupling LayersTraining the RealNVP ModelAnalysis of the RealNVP ModelOther Normalizing Flow ModelsGLOWFFJORDSummary

IntroductionEnergy-Based ModelsThe MNIST DatasetThe Energy FunctionSampling Using Langevin DynamicsTraining with Contrastive DivergenceAnalysis of the Energy-Based ModelOther Energy-Based ModelsSummary
IntroductionDenoising Diffusion Models (DDM)The Flowers DatasetThe Forward Diffusion ProcessThe Reparameterization TrickDiffusion SchedulesThe Reverse Diffusion ProcessThe U-Net Denoising ModelTraining the Diffusion ModelSampling from the Denoising Diffusion ModelAnalysis of the Diffusion ModelSummary
IntroductionGPTThe Wine Reviews DatasetAttentionQueries, Keys, and ValuesMultihead AttentionCausal MaskingThe Transformer BlockPositional EncodingTraining GPTAnalysis of GPTOther TransformersT5GPT-3 and GPT-4ChatGPTSummary
IntroductionProGANProgressive TrainingOutputsStyleGANThe Mapping NetworkThe Synthesis NetworkOutputs from StyleGANStyleGAN2Weight Modulation and DemodulationPath Length RegularizationNo Progressive GrowingOutputs from StyleGAN2Other Important GANsSelf-Attention GAN (SAGAN)BigGANVQ-GANViT VQ-GANSummary
IntroductionTransformers for Music GenerationThe Bach Cello Suite DatasetParsing MIDI FilesTokenizationCreating the Training SetSine Position EncodingMultiple Inputs and OutputsAnalysis of the Music-Generating TransformerTokenization of Polyphonic MusicMuseGANThe Bach Chorale DatasetThe MuseGAN GeneratorThe MuseGAN CriticAnalysis of the MuseGANSummary
IntroductionReinforcement LearningThe CarRacing EnvironmentWorld Model OverviewArchitectureTrainingCollecting Random Rollout DataTraining the VAEThe VAE ArchitectureExploring the VAECollecting Data to Train the MDN-RNNTraining the MDN-RNNThe MDN-RNN ArchitectureSampling from the MDN-RNNTraining the ControllerThe Controller ArchitectureCMA-ESParallelizing CMA-ESIn-Dream TrainingSummary
IntroductionDALL.E 2ArchitectureThe Text EncoderCLIPThe PriorThe DecoderExamples from DALL.E 2ImagenArchitectureDrawBenchExamples from ImagenStable DiffusionArchitectureExamples from Stable DiffusionFlamingoArchitectureThe Vision EncoderThe Perceiver ResamplerThe Language ModelExamples from FlamingoSummary
Timeline of Generative AI2014–2017: The VAE and GAN Era2018–2019: The Transformer Era2020–2022: The Big Model EraThe Current State of Generative AILarge Language ModelsText-to-Code ModelsText-to-Image ModelsOther ApplicationsThe Future of Generative AIGenerative AI in Everyday LifeGenerative AI in the WorkplaceGenerative AI in EducationGenerative AI Ethics and ChallengesFinal Thoughts

Overview

Generative AI is the hottest topic in tech. This practical book teaches machine learning engineers and data scientists how to use TensorFlow and Keras to create impressive generative deep learning models from scratch, including variational autoencoders (VAEs), generative adversarial networks (GANs), Transformers, normalizing flows, energy-based models, and denoising diffusion models.

The book starts with the basics of deep learning and progresses to cutting-edge architectures. Through tips and tricks, you'll understand how to make your models learn more efficiently and become more creative.

Discover how VAEs can change facial expressions in photos
Train GANs to generate images based on your own dataset
Build diffusion models to produce new varieties of flowers
Train your own GPT for text generation
Learn how large language models like ChatGPT are trained
Explore state-of-the-art architectures such as StyleGAN2 and ViT-VQGAN
Compose polyphonic music using Transformers and MuseGAN
Understand how generative world models can solve reinforcement learning tasks
Dive into multimodal models such as DALL.E 2, Imagen, and Stable Diffusion

This book also explores the future of generative AI and how individuals and companies can proactively begin to leverage this remarkable new technology to create competitive advantage.