book

Generative Deep Learning, 2nd Edition

by David Foster

April 2023

Intermediate to advanced

456 pages

11h 12m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

Objective and ApproachPrerequisitesRoadmapChanges in the Second EditionOther ResourcesConventions Used in This BookCodebaseUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Generative Modeling?Generative Versus Discriminative ModelingThe Rise of Generative ModelingGenerative Modeling and AIOur First Generative ModelHello World!The Generative Modeling FrameworkRepresentation LearningCore Probability TheoryGenerative Model TaxonomyThe Generative Deep Learning CodebaseCloning the RepositoryUsing DockerRunning on a GPUSummary
Data for Deep LearningDeep Neural NetworksWhat Is a Neural Network?Learning High-Level FeaturesTensorFlow and KerasMultilayer Perceptron (MLP)Preparing the DataBuilding the ModelCompiling the ModelTraining the ModelEvaluating the ModelConvolutional Neural Network (CNN)Convolutional LayersBatch NormalizationDropoutBuilding the CNNTraining and Evaluating the CNNSummary
IntroductionAutoencodersThe Fashion-MNIST DatasetThe Autoencoder ArchitectureThe EncoderThe DecoderJoining the Encoder to the DecoderReconstructing ImagesVisualizing the Latent SpaceGenerating New ImagesVariational AutoencodersThe EncoderThe Loss FunctionTraining the Variational AutoencoderAnalysis of the Variational AutoencoderExploring the Latent SpaceThe CelebA DatasetTraining the Variational AutoencoderAnalysis of the Variational AutoencoderGenerating New FacesLatent Space ArithmeticMorphing Between FacesSummary
IntroductionDeep Convolutional GAN (DCGAN)The Bricks DatasetThe DiscriminatorThe GeneratorTraining the DCGANAnalysis of the DCGANGAN Training: Tips and TricksWasserstein GAN with Gradient Penalty (WGAN-GP)Wasserstein LossThe Lipschitz ConstraintEnforcing the Lipschitz ConstraintThe Gradient Penalty LossTraining the WGAN-GPAnalysis of the WGAN-GPConditional GAN (CGAN)CGAN ArchitectureTraining the CGANAnalysis of the CGANSummary
IntroductionLong Short-Term Memory Network (LSTM)The Recipes DatasetWorking with Text DataTokenizationCreating the Training SetThe LSTM ArchitectureThe Embedding LayerThe LSTM LayerThe LSTM CellTraining the LSTMAnalysis of the LSTMRecurrent Neural Network (RNN) ExtensionsStacked Recurrent NetworksGated Recurrent UnitsBidirectional CellsPixelCNNMasked Convolutional LayersResidual BlocksTraining the PixelCNNAnalysis of the PixelCNNMixture DistributionsSummary
IntroductionNormalizing FlowsChange of VariablesThe Jacobian DeterminantThe Change of Variables EquationRealNVPThe Two Moons DatasetCoupling LayersTraining the RealNVP ModelAnalysis of the RealNVP ModelOther Normalizing Flow ModelsGLOWFFJORDSummary

IntroductionEnergy-Based ModelsThe MNIST DatasetThe Energy FunctionSampling Using Langevin DynamicsTraining with Contrastive DivergenceAnalysis of the Energy-Based ModelOther Energy-Based ModelsSummary
IntroductionDenoising Diffusion Models (DDM)The Flowers DatasetThe Forward Diffusion ProcessThe Reparameterization TrickDiffusion SchedulesThe Reverse Diffusion ProcessThe U-Net Denoising ModelTraining the Diffusion ModelSampling from the Denoising Diffusion ModelAnalysis of the Diffusion ModelSummary
IntroductionGPTThe Wine Reviews DatasetAttentionQueries, Keys, and ValuesMultihead AttentionCausal MaskingThe Transformer BlockPositional EncodingTraining GPTAnalysis of GPTOther TransformersT5GPT-3 and GPT-4ChatGPTSummary
IntroductionProGANProgressive TrainingOutputsStyleGANThe Mapping NetworkThe Synthesis NetworkOutputs from StyleGANStyleGAN2Weight Modulation and DemodulationPath Length RegularizationNo Progressive GrowingOutputs from StyleGAN2Other Important GANsSelf-Attention GAN (SAGAN)BigGANVQ-GANViT VQ-GANSummary
IntroductionTransformers for Music GenerationThe Bach Cello Suite DatasetParsing MIDI FilesTokenizationCreating the Training SetSine Position EncodingMultiple Inputs and OutputsAnalysis of the Music-Generating TransformerTokenization of Polyphonic MusicMuseGANThe Bach Chorale DatasetThe MuseGAN GeneratorThe MuseGAN CriticAnalysis of the MuseGANSummary
IntroductionReinforcement LearningThe CarRacing EnvironmentWorld Model OverviewArchitectureTrainingCollecting Random Rollout DataTraining the VAEThe VAE ArchitectureExploring the VAECollecting Data to Train the MDN-RNNTraining the MDN-RNNThe MDN-RNN ArchitectureSampling from the MDN-RNNTraining the ControllerThe Controller ArchitectureCMA-ESParallelizing CMA-ESIn-Dream TrainingSummary
IntroductionDALL.E 2ArchitectureThe Text EncoderCLIPThe PriorThe DecoderExamples from DALL.E 2ImagenArchitectureDrawBenchExamples from ImagenStable DiffusionArchitectureExamples from Stable DiffusionFlamingoArchitectureThe Vision EncoderThe Perceiver ResamplerThe Language ModelExamples from FlamingoSummary
Timeline of Generative AI2014–2017: The VAE and GAN Era2018–2019: The Transformer Era2020–2022: The Big Model EraThe Current State of Generative AILarge Language ModelsText-to-Code ModelsText-to-Image ModelsOther ApplicationsThe Future of Generative AIGenerative AI in Everyday LifeGenerative AI in the WorkplaceGenerative AI in EducationGenerative AI Ethics and ChallengesFinal Thoughts

Content preview from Generative Deep Learning, 2nd Edition

Chapter 14. Conclusion

In May 2018, I began work on the first edition of this book. Five years later, I am more excited than ever about the endless possibilities and potential impact of generative AI.

In this time we have seen incredible progress in this field, with seemingly limitless potential for real-world applications. I am filled with a sense of awe and wonder at what we have been able to achieve so far and eagerly anticipate witnessing the effect that generative AI will have on the world in the coming years. Generative deep learning has the power to shape the future in ways we can’t even begin to imagine.

What’s more, as I have been researching content for this book, it has become ever clearer to me that this field isn’t just about creating images, text, or music. I believe that at the core of generative deep learning lies the secret ...