book

Generative Deep Learning

by David Foster

June 2019

Intermediate to advanced

327 pages

7h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Objective and ApproachPrerequisitesOther ResourcesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Generative Modeling?Generative Versus Discriminative ModelingAdvances in Machine LearningThe Rise of Generative ModelingThe Generative Modeling FrameworkProbabilistic Generative ModelsHello Wrodl!Your First Probabilistic Generative ModelNaive BayesHello Wrodl! ContinuedThe Challenges of Generative ModelingRepresentation LearningSetting Up Your EnvironmentSummary
Structured and Unstructured DataDeep Neural NetworksKeras and TensorFlowYour First Deep Neural NetworkLoading the DataBuilding the ModelCompiling the ModelTraining the ModelEvaluating the ModelImproving the ModelConvolutional LayersBatch NormalizationDropout LayersPutting It All TogetherSummary
The Art ExhibitionAutoencodersYour First AutoencoderThe EncoderThe DecoderJoining the Encoder to the DecoderAnalysis of the AutoencoderThe Variational Art ExhibitionBuilding a Variational AutoencoderThe EncoderThe Loss FunctionAnalysis of the Variational AutoencoderUsing VAEs to Generate FacesTraining the VAEAnalysis of the VAEGenerating New FacesLatent Space ArithmeticMorphing Between FacesSummary
GanimalsIntroduction to GANsYour First GANThe DiscriminatorThe GeneratorTraining the GANGAN ChallengesOscillating LossMode CollapseUninformative LossHyperparametersTackling the GAN ChallengesWasserstein GANWasserstein LossThe Lipschitz ConstraintWeight ClippingTraining the WGANAnalysis of the WGANWGAN-GPThe Gradient Penalty LossAnalysis of WGAN-GPSummary
Apples and OrgangesCycleGANYour First CycleGANOverviewThe Generators (U-Net)The DiscriminatorsCompiling the CycleGANTraining the CycleGANAnalysis of the CycleGANCreating a CycleGAN to Paint Like MonetThe Generators (ResNet)Analysis of the CycleGANNeural Style TransferContent LossStyle LossTotal Variance LossRunning the Neural Style TransferAnalysis of the Neural Style Transfer ModelSummary
The Literary Society for Troublesome MiscreantsLong Short-Term Memory NetworksYour First LSTM NetworkTokenizationBuilding the DatasetThe LSTM ArchitectureThe Embedding LayerThe LSTM LayerThe LSTM CellGenerating New TextRNN ExtensionsStacked Recurrent NetworksGated Recurrent UnitsBidirectional CellsEncoder–Decoder ModelsA Question and Answer GeneratorA Question-Answer DatasetModel ArchitectureInferenceModel ResultsSummary
PreliminariesMusical NotationYour First Music-Generating RNNAttentionBuilding an Attention Mechanism in KerasAnalysis of the RNN with AttentionAttention in Encoder–Decoder NetworksGenerating Polyphonic MusicThe Musical OrganYour First MuseGANThe MuseGAN GeneratorChords, Style, Melody, and GrooveThe Bar GeneratorPutting It All TogetherThe CriticAnalysis of the MuseGANSummary

Reinforcement LearningOpenAI GymWorld Model ArchitectureThe Variational AutoencoderThe MDN-RNNThe ControllerSetupTraining Process OverviewCollecting Random Rollout DataTraining the VAEThe VAE ArchitectureExploring the VAECollecting Data to Train the RNNTraining the MDN-RNNThe MDN-RNN ArchitectureSampling the Next z and Reward from the MDN-RNNThe MDN-RNN Loss FunctionTraining the ControllerThe Controller ArchitectureCMA-ESParallelizing CMA-ESOutput from the Controller TrainingIn-Dream TrainingIn-Dream Training the ControllerChallenges of In-Dream TrainingSummary
Five Years of ProgressThe TransformerPositional EncodingMultihead AttentionThe DecoderAnalysis of the TransformerBERTGPT-2MuseNetAdvances in Image GenerationProGANSelf-Attention GAN (SAGAN)BigGANStyleGANApplications of Generative ModelingAI ArtAI Music

Content preview from Generative Deep Learning

Chapter 4. Generative Adversarial Networks

On Monday, December 5, 2016, at 2:30 p.m., Ian Goodfellow of Google Brain presented a tutorial entitled “Generative Adversarial Networks” to the delegates of the Neural Information Processing Systems (NIPS) conference in Barcelona.¹ The ideas presented in the tutorial are now regarded as one of the key turning points for generative modeling and have spawned a wide variety of variations on his core idea that have pushed the field to even greater heights.

This chapter will first lay out the theoretical underpinning of generative adversarial networks (GANs). You will then learn how to use the Python library Keras to start building your own GANs.

First though, we shall take a trip into the wilderness to meet Gene…

Ganimals

One afternoon, while walking through the local jungle, Gene sees a woman thumbing through a set of black and white photographs, looking worried. He goes over to ask if he can help.

The woman introduces herself as Di, a keen explorer, and explains that she is hunting for the elusive ganimal, a mythical creature that is said to roam around the jungle. Since the creature is nocturnal, she only has a collection of nighttime photos of the beast that she once found lying on the floor of the jungle, dropped by another ganimal enthusiast. Some of these photos are shown in Figure 4-1. Di makes money by selling the images to collectors but is starting to worry, as she hasn’t actually ever seen the creatures and is concerned that ...