book

Generative Deep Learning

by David Foster

June 2019

Intermediate to advanced

327 pages

7h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Objective and ApproachPrerequisitesOther ResourcesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Generative Modeling?Generative Versus Discriminative ModelingAdvances in Machine LearningThe Rise of Generative ModelingThe Generative Modeling FrameworkProbabilistic Generative ModelsHello Wrodl!Your First Probabilistic Generative ModelNaive BayesHello Wrodl! ContinuedThe Challenges of Generative ModelingRepresentation LearningSetting Up Your EnvironmentSummary
Structured and Unstructured DataDeep Neural NetworksKeras and TensorFlowYour First Deep Neural NetworkLoading the DataBuilding the ModelCompiling the ModelTraining the ModelEvaluating the ModelImproving the ModelConvolutional LayersBatch NormalizationDropout LayersPutting It All TogetherSummary
The Art ExhibitionAutoencodersYour First AutoencoderThe EncoderThe DecoderJoining the Encoder to the DecoderAnalysis of the AutoencoderThe Variational Art ExhibitionBuilding a Variational AutoencoderThe EncoderThe Loss FunctionAnalysis of the Variational AutoencoderUsing VAEs to Generate FacesTraining the VAEAnalysis of the VAEGenerating New FacesLatent Space ArithmeticMorphing Between FacesSummary
GanimalsIntroduction to GANsYour First GANThe DiscriminatorThe GeneratorTraining the GANGAN ChallengesOscillating LossMode CollapseUninformative LossHyperparametersTackling the GAN ChallengesWasserstein GANWasserstein LossThe Lipschitz ConstraintWeight ClippingTraining the WGANAnalysis of the WGANWGAN-GPThe Gradient Penalty LossAnalysis of WGAN-GPSummary
Apples and OrgangesCycleGANYour First CycleGANOverviewThe Generators (U-Net)The DiscriminatorsCompiling the CycleGANTraining the CycleGANAnalysis of the CycleGANCreating a CycleGAN to Paint Like MonetThe Generators (ResNet)Analysis of the CycleGANNeural Style TransferContent LossStyle LossTotal Variance LossRunning the Neural Style TransferAnalysis of the Neural Style Transfer ModelSummary
The Literary Society for Troublesome MiscreantsLong Short-Term Memory NetworksYour First LSTM NetworkTokenizationBuilding the DatasetThe LSTM ArchitectureThe Embedding LayerThe LSTM LayerThe LSTM CellGenerating New TextRNN ExtensionsStacked Recurrent NetworksGated Recurrent UnitsBidirectional CellsEncoder–Decoder ModelsA Question and Answer GeneratorA Question-Answer DatasetModel ArchitectureInferenceModel ResultsSummary
PreliminariesMusical NotationYour First Music-Generating RNNAttentionBuilding an Attention Mechanism in KerasAnalysis of the RNN with AttentionAttention in Encoder–Decoder NetworksGenerating Polyphonic MusicThe Musical OrganYour First MuseGANThe MuseGAN GeneratorChords, Style, Melody, and GrooveThe Bar GeneratorPutting It All TogetherThe CriticAnalysis of the MuseGANSummary

Reinforcement LearningOpenAI GymWorld Model ArchitectureThe Variational AutoencoderThe MDN-RNNThe ControllerSetupTraining Process OverviewCollecting Random Rollout DataTraining the VAEThe VAE ArchitectureExploring the VAECollecting Data to Train the RNNTraining the MDN-RNNThe MDN-RNN ArchitectureSampling the Next z and Reward from the MDN-RNNThe MDN-RNN Loss FunctionTraining the ControllerThe Controller ArchitectureCMA-ESParallelizing CMA-ESOutput from the Controller TrainingIn-Dream TrainingIn-Dream Training the ControllerChallenges of In-Dream TrainingSummary
Five Years of ProgressThe TransformerPositional EncodingMultihead AttentionThe DecoderAnalysis of the TransformerBERTGPT-2MuseNetAdvances in Image GenerationProGANSelf-Attention GAN (SAGAN)BigGANStyleGANApplications of Generative ModelingAI ArtAI Music

Content preview from Generative Deep Learning

Chapter 1. Generative Modeling

This chapter is a general introduction to the field of generative modeling. We shall first look at what it means to say that a model is generative and learn how it differs from the more widely studied discriminative modeling. Then I will introduce the framework and core mathematical ideas that will allow us to structure our general approach to problems that require a generative solution.

With this in place, we will then build our first example of a generative model (Naive Bayes) that is probabilistic in nature. We shall see that this allows us to generate novel examples that are outside of our training dataset, but shall also explore the reasons why this type of model may fail as the size and complexity of the space of possible creations increases.

What Is Generative Modeling?

A generative model can be broadly defined as follows:

A generative model describes how a dataset is generated, in terms of a probabilistic model. By sampling from this model, we are able to generate new data.

Suppose we have a dataset containing images of horses. We may wish to build a model that can generate a new image of a horse that has never existed but still looks real because the model has learned the general rules that govern the appearance of a horse. This is the kind of problem that can be solved using generative modeling. A summary of a typical generative modeling process is shown in Figure 1-1.

First, we require a dataset consisting of many examples of the entity ...