book

Generative Deep Learning

by David Foster

June 2019

Intermediate to advanced

327 pages

7h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Objective and ApproachPrerequisitesOther ResourcesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Generative Modeling?Generative Versus Discriminative ModelingAdvances in Machine LearningThe Rise of Generative ModelingThe Generative Modeling FrameworkProbabilistic Generative ModelsHello Wrodl!Your First Probabilistic Generative ModelNaive BayesHello Wrodl! ContinuedThe Challenges of Generative ModelingRepresentation LearningSetting Up Your EnvironmentSummary
Structured and Unstructured DataDeep Neural NetworksKeras and TensorFlowYour First Deep Neural NetworkLoading the DataBuilding the ModelCompiling the ModelTraining the ModelEvaluating the ModelImproving the ModelConvolutional LayersBatch NormalizationDropout LayersPutting It All TogetherSummary
The Art ExhibitionAutoencodersYour First AutoencoderThe EncoderThe DecoderJoining the Encoder to the DecoderAnalysis of the AutoencoderThe Variational Art ExhibitionBuilding a Variational AutoencoderThe EncoderThe Loss FunctionAnalysis of the Variational AutoencoderUsing VAEs to Generate FacesTraining the VAEAnalysis of the VAEGenerating New FacesLatent Space ArithmeticMorphing Between FacesSummary
GanimalsIntroduction to GANsYour First GANThe DiscriminatorThe GeneratorTraining the GANGAN ChallengesOscillating LossMode CollapseUninformative LossHyperparametersTackling the GAN ChallengesWasserstein GANWasserstein LossThe Lipschitz ConstraintWeight ClippingTraining the WGANAnalysis of the WGANWGAN-GPThe Gradient Penalty LossAnalysis of WGAN-GPSummary
Apples and OrgangesCycleGANYour First CycleGANOverviewThe Generators (U-Net)The DiscriminatorsCompiling the CycleGANTraining the CycleGANAnalysis of the CycleGANCreating a CycleGAN to Paint Like MonetThe Generators (ResNet)Analysis of the CycleGANNeural Style TransferContent LossStyle LossTotal Variance LossRunning the Neural Style TransferAnalysis of the Neural Style Transfer ModelSummary
The Literary Society for Troublesome MiscreantsLong Short-Term Memory NetworksYour First LSTM NetworkTokenizationBuilding the DatasetThe LSTM ArchitectureThe Embedding LayerThe LSTM LayerThe LSTM CellGenerating New TextRNN ExtensionsStacked Recurrent NetworksGated Recurrent UnitsBidirectional CellsEncoder–Decoder ModelsA Question and Answer GeneratorA Question-Answer DatasetModel ArchitectureInferenceModel ResultsSummary
PreliminariesMusical NotationYour First Music-Generating RNNAttentionBuilding an Attention Mechanism in KerasAnalysis of the RNN with AttentionAttention in Encoder–Decoder NetworksGenerating Polyphonic MusicThe Musical OrganYour First MuseGANThe MuseGAN GeneratorChords, Style, Melody, and GrooveThe Bar GeneratorPutting It All TogetherThe CriticAnalysis of the MuseGANSummary

Reinforcement LearningOpenAI GymWorld Model ArchitectureThe Variational AutoencoderThe MDN-RNNThe ControllerSetupTraining Process OverviewCollecting Random Rollout DataTraining the VAEThe VAE ArchitectureExploring the VAECollecting Data to Train the RNNTraining the MDN-RNNThe MDN-RNN ArchitectureSampling the Next z and Reward from the MDN-RNNThe MDN-RNN Loss FunctionTraining the ControllerThe Controller ArchitectureCMA-ESParallelizing CMA-ESOutput from the Controller TrainingIn-Dream TrainingIn-Dream Training the ControllerChallenges of In-Dream TrainingSummary
Five Years of ProgressThe TransformerPositional EncodingMultihead AttentionThe DecoderAnalysis of the TransformerBERTGPT-2MuseNetAdvances in Image GenerationProGANSelf-Attention GAN (SAGAN)BigGANStyleGANApplications of Generative ModelingAI ArtAI Music

Content preview from Generative Deep Learning

Chapter 8. Play

In March 2018, David Ha and Jürgen Schmidhuber published their “World Models” paper.¹ The paper showed how it is possible to train a model that can learn how to perform a particular task through experimentation within its own generative hallucinated dreams, rather than inside the environment itself. It is an excellent example of how generative modeling can be used to solve practical problems, when applied alongside other machine learning techniques such as reinforcement learning.

A key component of the architecture is a generative model that can construct a probability distribution for the next possible state, given the current state and action. Having built up an understanding of the underlying physics of the environment through random movements, the model is then able to train itself from scratch on a new task, entirely within its own internal representation of the environment. This approach led to world-best scores for both of the tasks on which it was tested.

In this chapter, we will explore the model in detail and show how it is possible to create your own version of this amazing cutting-edge technology.

Based on the original paper, we will be building a reinforcement learning algorithm that learns how to drive a car around a racetrack as fast as possible. While we will be using a 2D computer simulation as our environment, the same technique could also be applied to real-world scenarios where testing strategies in the live environment is expensive or infeasible. ...