Hands-On Deep Learning for Games

Hands-On Deep Learning for Games

by Micheal Lanham
Released March 2019
Publisher(s): Packt Publishing
ISBN: 9781788994071

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Understand the core concepts of deep learning and deep reinforcement learning by applying them to develop games

Key Features

  • Apply the power of deep learning to complex reasoning tasks by building a Game AI
  • Exploit the most recent developments in machine learning and AI for building smart games
  • Implement deep learning models and neural networks with Python

Book Description

The number of applications of deep learning and neural networks has multiplied in the last couple of years. Neural nets has enabled significant breakthroughs in everything from computer vision, voice generation, voice recognition and self-driving cars. Game development is also a key area where these techniques are being applied. This book will give an in depth view of the potential of deep learning and neural networks in game development.

We will take a look at the foundations of multi-layer perceptron's to using convolutional and recurrent networks. In applications from GANs that create music or textures to self-driving cars and chatbots. Then we introduce deep reinforcement learning through the multi-armed bandit problem and other OpenAI Gym environments.

As we progress through the book we will gain insights about DRL techniques such as Motivated Reinforcement Learning with Curiosity and Curriculum Learning. We also take a closer look at deep reinforcement learning and in particular the Unity ML-Agents toolkit. By the end of the book, we will look at how to apply DRL and the ML-Agents toolkit to enhance, test and automate your games or simulations. Finally, we will cover your possible next steps and possible areas for future learning.

What you will learn

  • Learn the foundations of neural networks and deep learning.
  • Use advanced neural network architectures in applications to create music, textures, self driving cars and chatbots.
  • Understand the basics of reinforcement and DRL and how to apply it to solve a variety of problems.
  • Working with Unity ML-Agents toolkit and how to install, setup and run the kit.
  • Understand core concepts of DRL and the differences between discrete and continuous action environments.
  • Use several advanced forms of learning in various scenarios from developing agents to testing games.

Who this book is for

This books is for game developers who wish to create highly interactive games by leveraging the power of machine and deep learning. No prior knowledge of machine learning, deep learning or neural networks is required this book will teach those concepts from scratch. A good understanding of Python is required.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Hands-On Deep Learning for Games
  3. Dedication
  4. About Packt
    1. Why subscribe?
    2. Packt.com
  5. Contributors
    1. About the author
    2. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  7. Section 1: The Basics
  8. Deep Learning for Games
    1. The past, present, and future of DL
      1. The past
      2. The present
      3. The future
    2. Neural networks – the foundation
      1. Training a perceptron in Python
    3. Multilayer perceptron in TF
    4. TensorFlow Basics
    5. Training neural networks with backpropagation
      1. The Cost function
      2. Partial differentiation and the chain rule
    6. Building an autoencoder with Keras
      1. Training the model
      2. Examining the output
    7. Exercises
    8. Summary
  9. Convolutional and Recurrent Networks
    1. Convolutional neural networks
      1. Monitoring training with TensorBoard
    2. Understanding convolution
    3. Building a self-driving CNN
      1. Spatial convolution and pooling
      2. The need for Dropout
    4. Memory and recurrent networks
      1. Vanishing and exploding gradients rescued by LSTM
    5. Playing Rock, Paper, Scissors with LSTMs
    6. Exercises
    7. Summary
  10. GAN for Games
    1. Introducing GANs
    2. Coding a GAN in Keras
      1. Training a GAN
      2. Optimizers
    3. Wasserstein GAN
    4. Generating textures with a GAN 
      1. Batch normalization
      2. Leaky and other ReLUs
    5. A GAN for creating music
      1. Training the music GAN
      2. Generating music via an alternative GAN
    6. Exercises
    7. Summary 
  11. Building a Deep Learning Gaming Chatbot
    1. Neural conversational agents
      1. General conversational models
    2. Sequence-to-sequence learning
      1. Breaking down the code
      2. Thought vectors
    3. DeepPavlov
    4. Building the chatbot server
      1. Message hubs (RabbitMQ)
      2. Managing RabbitMQ
      3. Sending and receiving to/from the MQ
      4. Writing the message queue chatbot
    5. Running the chatbot in Unity
      1. Installing AMQP for Unity
    6. Exercises
    7. Summary
  12. Section 2: Deep Reinforcement Learning
  13. Introducing DRL
    1. Reinforcement learning
      1. The multi-armed bandit
      2. Contextual bandits
    2. RL with the OpenAI Gym
    3. A Q-Learning model
      1. Markov decision process and the Bellman equation
      2. Q-learning
      3. Q-learning and exploration
    4. First DRL with Deep Q-learning
    5. RL experiments
      1. Keras RL
    6. Exercises
    7. Summary
  14. Unity ML-Agents
    1. Installing ML-Agents
    2. Training an agent
    3. What's in a brain?
    4. Monitoring training with TensorBoard
    5. Running an agent
      1. Loading a trained brain
    6. Exercises
    7. Summary
  15. Agent and the Environment
    1. Exploring the training environment
      1. Training the agent visually
      2. Reverting to the basics
    2. Understanding state
    3. Understanding visual state
    4. Convolution and visual state
      1. To pool or not to pool
    5. Recurrent networks for remembering series
      1. Tuning recurrent hyperparameters
    6. Exercises
    7. Summary
  16. Understanding PPO
    1. Marathon RL
    2. The partially observable Markov decision process
    3. Actor-Critic and continuous action spaces
      1. Expanding network architecture
    4. Understanding TRPO and PPO
      1. Generalized advantage estimate
    5. Learning to tune PPO 
      1. Coding changes required for control projects
      2. Multiple agent policy
    6. Exercises 
    7. Summary
  17. Rewards and Reinforcement Learning
    1. Rewards and reward functions
      1. Building reward functions
    2. Sparsity of rewards
    3. Curriculum Learning
    4. Understanding Backplay
      1. Implementing Backplay through Curriculum Learning
    5. Curiosity Learning
      1. The Curiosity Intrinsic module in action
      2. Trying ICM on Hallway/VisualHallway
    6. Exercises
    7. Summary
  18. Imitation and Transfer Learning
    1. IL, or behavioral cloning
    2. Online training
    3. Offline training
      1. Setting up for training
      2. Feeding the agent
    4. Transfer learning
      1. Transferring a brain
      2. Exploring TensorFlow checkpoints
    5. Imitation Transfer Learning
      1. Training multiple agents with one demonstration
    6. Exercises
    7. Summary
  19. Building Multi-Agent Environments
    1. Adversarial and cooperative self-play
      1. Training self-play environments
    2. Adversarial self-play
    3. Multi-brain play
    4. Adding individuality with intrinsic rewards
    5. Extrinsic rewards for individuality
      1. Creating uniqueness with customized reward functions 
      2. Configuring the agents' personalities
    6. Exercises
    7. Summary
  20. Section 3: Building Games
  21. Debugging/Testing a Game with DRL
    1. Introducing the game
    2. Setting up ML-Agents
      1. Introducing rewards to the game
      2. Setting up TestingAcademy
      3. Scripting the TestingAgent
      4. Setting up the TestingAgent
    3. Overriding the Unity input system
      1. Building the TestingInput
      2. Adding TestingInput to the scene
      3. Overriding the game input
      4. Configuring the required brains
      5. Time for training
    4. Testing through imitation
      1. Configuring the agent to use IL
    5. Analyzing the testing process
      1. Sending custom analytics
    6. Exercises
    7. Summary
  22. Obstacle Tower Challenge and Beyond
    1. The Unity Obstacle Tower Challenge
    2. Deep Learning for your game?
    3. Building your game 
    4. More foundations of learning
    5. Summary
  23. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Hands-On Deep Learning for Games
  • Author(s): Micheal Lanham
  • Release date: March 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781788994071