book

Evolutionary Deep Learning

by Micheal Lanham

August 2023

Intermediate to advanced

360 pages

10h 23m

English

Manning Publications

Read now

Unlock full access

prefaceacknowledgmentsabout this bookWho should read this bookHow this book is organized: A road mapAbout the codeliveBook discussion forumabout the authorabout the cover illustration
1.1 What is evolutionary deep learning?1.1.1 Introducing evolutionary computation1.2 The why and where of evolutionary deep learning1.3 The need for deep learning optimization1.3.1 Optimizing the network architecture1.4 Automating optimization with automated machine learning1.4.1 What is automated machine learning?1.5 Applications of evolutionary deep learning1.5.1 Model selection: Weight search1.5.2 Model architecture: Architecture optimization1.5.3 Hyperparameter tuning/optimization1.5.4 Validation and loss function optimization1.5.5 Neuroevolution of augmenting topologies1.5.6 GoalsSummary
2.1 Conway’s Game of Life on Google Colaboratory2.2 Simulating life with Python2.2.1 Learning exercises2.3 Life simulation as optimization2.3.1 Learning exercises2.4 Adding evolution to the life simulation2.4.1 Simulating evolution2.4.2 Learning exercises2.4.3 Some background on Darwin and evolution2.4.4 Natural selection and survival of the fittest2.5 Genetic algorithms in Python2.5.1 Understanding genetics and meiosis2.5.2 Coding genetic algorithms2.5.3 Constructing the population2.5.4 Evaluating fitness2.5.5 Selecting for reproduction (crossover)2.5.6 Applying crossover: Reproduction2.5.7 Applying mutation and variation2.5.8 Putting it all together2.5.9 Understanding genetic algorithm hyperparameters2.5.10 Learning exercisesSummary
3.1 Genetic algorithms in DEAP3.1.1 One max with DEAP3.1.2 Learning exercises3.2 Solving the Queen’s Gambit3.2.1 Learning exercises3.3 Helping a traveling salesman3.3.1 Building the TSP solver3.3.2 Learning exercises3.4 Selecting genetic operators for improved evolution3.4.1 Learning exercises3.5 Painting with the EvoLisa3.5.1 Learning exercisesSummary

4.1 Genetic programming with DEAP4.1.1 Solving regression with genetic programming4.1.2 Learning exercises4.2 Particle swarm optimization with DEAP4.2.1 Solving equations with PSO4.2.2 Learning exercises4.3 Coevolving solutions with DEAP4.3.1 Coevolving genetic programming with genetic algorithms4.4 Evolutionary strategies with DEAP4.4.1 Applying evolutionary strategies to function approximation4.4.2 Revisiting the EvoLisa4.4.3 Learning exercises4.5 Differential evolution with DEAP4.5.1 Approximating complex and discontinuous functions with DE4.5.2 Learning exercisesSummary
5.1 Option selection and hyperparameter tuning5.1.1 Tuning hyperparameter strategies5.1.2 Selecting model options5.2 Automating HPO with random search5.2.1 Applying random search to HPO5.3 Grid search and HPO5.3.1 Using grid search for automatic HPO5.4 Evolutionary computation for HPO5.4.1 Particle swarm optimization for HPO5.4.2 Adding EC and DEAP to automatic HPO5.5 Genetic algorithms and evolutionary strategies for HPO5.5.1 Applying evolutionary strategies to HPO5.5.2 Expanding dimensions with principal component analysis5.6 Differential evolution for HPO5.6.1 Differential search for evolving HPOSummary
6.1 Multilayered perceptron in NumPy6.1.1 Learning exercises6.2 Genetic algorithms as deep learning optimizers6.2.1 Learning exercises6.3 Other evolutionary methods for neurooptimization6.3.1 Learning exercises6.4 Applying neuroevolution optimization to Keras6.4.1 Learning exercises6.5 Understanding the limits of evolutionary optimization6.5.1 Learning exercisesSummary
7.1 Reviewing convolutional neural networks in Keras7.1.1 Understanding CNN layer problems7.1.2 Learning exercises7.2 Encoding a network architecture in genes7.2.1 Learning exercises7.3 Creating the mating crossover operation7.4 Developing a custom mutation operator7.5 Evolving convolutional network architecture7.5.1 Learning exercisesSummary
8.1 The convolution autoencoder8.1.1 Introducing autoencoders8.1.2 Building a convolutional autoencoder8.1.3 Learning exercises8.1.4 Generalizing a convolutional AE8.1.5 Improving the autoencoder8.2 Evolutionary AE optimization8.2.1 Building the AE gene sequence8.2.2 Learning exercises8.3 Mating and mutating the autoencoder gene sequence8.4 Evolving an autoencoder8.4.1 Learning exercises8.5 Building variational autoencoders8.5.1 Variational autoencoders: A review8.5.2 Implementing a VAE8.5.3 Learning exercisesSummary
9.1 Generative adversarial networks9.1.1 Introducing GANs9.1.2 Building a convolutional generative adversarial network in Keras9.1.3 Learning exercises9.2 The challenges of training a GAN9.2.1 The GAN optimization problem9.2.2 Observing vanishing gradients9.2.3 Observing mode collapse in GANs9.2.4 Observing convergence failures in GANs9.2.5 Learning exercises9.3 Fixing GAN problems with Wasserstein loss9.3.1 Understanding Wasserstein loss9.3.2 Improving the DCGAN with Wasserstein loss9.4 Encoding the Wasserstein DCGAN for evolution9.4.1 Learning exercises9.5 Optimizing the DCGAN with genetic algorithms9.5.1 Learning exercisesSummary
10.1 Exploring NEAT with NEAT-Python10.1.1 Learning exercises10.2 Visualizing an evolved NEAT network10.3 Exercising the capabilities of NEAT10.3.1 Learning exercises10.4 Exercising NEAT to classify images10.4.1 Learning exercises10.5 Uncovering the role of speciation in evolving topologies10.5.1 Tuning NEAT speciation10.5.2 Learning exercisesSummary
11.1 Introducing reinforcement learning11.1.1 Q-learning agent on the frozen lake11.1.2 Learning exercises11.2 Exploring complex problems from the OpenAI Gym11.2.1 Learning exercises11.3 Solving reinforcement learning problems with NEAT11.3.1 Learning exercises11.4 Solving Gym’s lunar lander problem with NEAT agents11.4.1 Learning exercises11.5 Solving Gym’s lunar lander problem with a deep Q-networkSummary
12.1 Evolution and machine learning with gene expression programming12.1.1 Learning exercises12.2 Revisiting reinforcement learning with Geppy12.2.1 Learning exercises12.3 Introducing instinctual learning12.3.1 The basics of instinctual learning12.3.2 Developing generalized instincts12.3.3 Evolving generalized solutions without instincts12.3.4 Learning exercises12.4 Generalized learning with genetic programming12.4.1 Learning exercises12.5 The future of evolutionary machine learning12.5.1 Is evolution broken?12.5.2 Evolutionary plasticity12.5.3 Improving evolution with plasticity12.5.4 Computation and evolutionary search12.6 Generalization with instinctual deep and deep reinforcement learningSummary
A.1 Accessing the source codeA.2 Running code on other platforms

Content preview from Evolutionary Deep Learning

11 Evolutionary learning with NEAT

This chapter covers

Introducing to reinforcement learning
Exploring complex problems from the OpenAI Gym
Using NEAT as an agent to solve reinforcement learning problems
Solving Gym’s lunar lander problem with a NEAT agent
Solving Gym’s lunar lander problem with a deep Q-network

In the last chapter, we explored NeuroEvolution of Augmenting Topologies (NEAT) to solve common problems we explored in previous chapters. In this chapter, we look at the evolution of learning itself. First, we use NEAT to develop an evolving agent that can solve problems typically associated with RL. Then, we look at more difficult RL problems and provide a NEAT solution for evolutionary learning. Finally, we finish the chapter by ...