book

Generative Deep Learning, 2nd Edition

by David Foster

April 2023

Intermediate to advanced

456 pages

11h 12m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

Objective and ApproachPrerequisitesRoadmapChanges in the Second EditionOther ResourcesConventions Used in This BookCodebaseUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Generative Modeling?Generative Versus Discriminative ModelingThe Rise of Generative ModelingGenerative Modeling and AIOur First Generative ModelHello World!The Generative Modeling FrameworkRepresentation LearningCore Probability TheoryGenerative Model TaxonomyThe Generative Deep Learning CodebaseCloning the RepositoryUsing DockerRunning on a GPUSummary
Data for Deep LearningDeep Neural NetworksWhat Is a Neural Network?Learning High-Level FeaturesTensorFlow and KerasMultilayer Perceptron (MLP)Preparing the DataBuilding the ModelCompiling the ModelTraining the ModelEvaluating the ModelConvolutional Neural Network (CNN)Convolutional LayersBatch NormalizationDropoutBuilding the CNNTraining and Evaluating the CNNSummary
IntroductionAutoencodersThe Fashion-MNIST DatasetThe Autoencoder ArchitectureThe EncoderThe DecoderJoining the Encoder to the DecoderReconstructing ImagesVisualizing the Latent SpaceGenerating New ImagesVariational AutoencodersThe EncoderThe Loss FunctionTraining the Variational AutoencoderAnalysis of the Variational AutoencoderExploring the Latent SpaceThe CelebA DatasetTraining the Variational AutoencoderAnalysis of the Variational AutoencoderGenerating New FacesLatent Space ArithmeticMorphing Between FacesSummary
IntroductionDeep Convolutional GAN (DCGAN)The Bricks DatasetThe DiscriminatorThe GeneratorTraining the DCGANAnalysis of the DCGANGAN Training: Tips and TricksWasserstein GAN with Gradient Penalty (WGAN-GP)Wasserstein LossThe Lipschitz ConstraintEnforcing the Lipschitz ConstraintThe Gradient Penalty LossTraining the WGAN-GPAnalysis of the WGAN-GPConditional GAN (CGAN)CGAN ArchitectureTraining the CGANAnalysis of the CGANSummary
IntroductionLong Short-Term Memory Network (LSTM)The Recipes DatasetWorking with Text DataTokenizationCreating the Training SetThe LSTM ArchitectureThe Embedding LayerThe LSTM LayerThe LSTM CellTraining the LSTMAnalysis of the LSTMRecurrent Neural Network (RNN) ExtensionsStacked Recurrent NetworksGated Recurrent UnitsBidirectional CellsPixelCNNMasked Convolutional LayersResidual BlocksTraining the PixelCNNAnalysis of the PixelCNNMixture DistributionsSummary
IntroductionNormalizing FlowsChange of VariablesThe Jacobian DeterminantThe Change of Variables EquationRealNVPThe Two Moons DatasetCoupling LayersTraining the RealNVP ModelAnalysis of the RealNVP ModelOther Normalizing Flow ModelsGLOWFFJORDSummary

IntroductionEnergy-Based ModelsThe MNIST DatasetThe Energy FunctionSampling Using Langevin DynamicsTraining with Contrastive DivergenceAnalysis of the Energy-Based ModelOther Energy-Based ModelsSummary
IntroductionDenoising Diffusion Models (DDM)The Flowers DatasetThe Forward Diffusion ProcessThe Reparameterization TrickDiffusion SchedulesThe Reverse Diffusion ProcessThe U-Net Denoising ModelTraining the Diffusion ModelSampling from the Denoising Diffusion ModelAnalysis of the Diffusion ModelSummary
IntroductionGPTThe Wine Reviews DatasetAttentionQueries, Keys, and ValuesMultihead AttentionCausal MaskingThe Transformer BlockPositional EncodingTraining GPTAnalysis of GPTOther TransformersT5GPT-3 and GPT-4ChatGPTSummary
IntroductionProGANProgressive TrainingOutputsStyleGANThe Mapping NetworkThe Synthesis NetworkOutputs from StyleGANStyleGAN2Weight Modulation and DemodulationPath Length RegularizationNo Progressive GrowingOutputs from StyleGAN2Other Important GANsSelf-Attention GAN (SAGAN)BigGANVQ-GANViT VQ-GANSummary
IntroductionTransformers for Music GenerationThe Bach Cello Suite DatasetParsing MIDI FilesTokenizationCreating the Training SetSine Position EncodingMultiple Inputs and OutputsAnalysis of the Music-Generating TransformerTokenization of Polyphonic MusicMuseGANThe Bach Chorale DatasetThe MuseGAN GeneratorThe MuseGAN CriticAnalysis of the MuseGANSummary
IntroductionReinforcement LearningThe CarRacing EnvironmentWorld Model OverviewArchitectureTrainingCollecting Random Rollout DataTraining the VAEThe VAE ArchitectureExploring the VAECollecting Data to Train the MDN-RNNTraining the MDN-RNNThe MDN-RNN ArchitectureSampling from the MDN-RNNTraining the ControllerThe Controller ArchitectureCMA-ESParallelizing CMA-ESIn-Dream TrainingSummary
IntroductionDALL.E 2ArchitectureThe Text EncoderCLIPThe PriorThe DecoderExamples from DALL.E 2ImagenArchitectureDrawBenchExamples from ImagenStable DiffusionArchitectureExamples from Stable DiffusionFlamingoArchitectureThe Vision EncoderThe Perceiver ResamplerThe Language ModelExamples from FlamingoSummary
Timeline of Generative AI2014–2017: The VAE and GAN Era2018–2019: The Transformer Era2020–2022: The Big Model EraThe Current State of Generative AILarge Language ModelsText-to-Code ModelsText-to-Image ModelsOther ApplicationsThe Future of Generative AIGenerative AI in Everyday LifeGenerative AI in the WorkplaceGenerative AI in EducationGenerative AI Ethics and ChallengesFinal Thoughts

Content preview from Generative Deep Learning, 2nd Edition

Foreword

This book is becoming part of my life. On finding a copy in my living room I asked my son, “When did you get this?” He replied, “When you gave it to me,” bemused by my senior moment. Going through various sections together, I came to regard Generative Deep Learning as the Gray’s Anatomy of Generative AI.

The author dissects the anatomy of generative AI with an incredible clarity and reassuring authority. He offers a truly remarkable account of a fast-moving field, underwritten with pragmatic examples, engaging narratives, and references that are so current, it reads like a living history.

Throughout his deconstructions, the author maintains a sense of wonder and excitement about the potential of generative AI—especially evident in the book’s compelling dénouement. Having laid bare the technology, he reminds us that we are at the dawn of a new age of intelligence, an age in which generative AI holds a mirror up to our language, our art, our creativity; reflecting not just what we have created, but what we could create—what we can create—limited only by “your own imagination.”

The central theme of generative models in artificial intelligence resonates deeply with me, because I see exactly the same themes emerging in the natural sciences; namely, a view of ourselves as generative models of our lived world. I suspect in the next edition of this book we will read about the confluence of artificial and natural intelligence. Until that time, I will keep this edition next to ...