Book description
We all learn through trial and error. We avoid the things that cause us to experience pain and failure. We embrace and build on the things that give us reward and success. This common pattern is the foundation of deep reinforcement learning: building machine learning systems that explore and learn based on the responses of the environment. Grokking Deep Reinforcement Learning introduces this powerful machine learning approach, using examples, illustrations, exercises, and crystal-clear teaching. You'll love the perfectly paced teaching and the clever, engaging writing style as you dig into this awesome exploration of reinforcement learning fundamentals, effective deep learning techniques, and practical applications in this emerging field.
About the Technology
We learn by interacting with our environment, and the rewards or punishments we experience guide our future behavior. Deep reinforcement learning brings that same natural process to artificial intelligence, analyzing results to uncover the most efficient ways forward. DRL agents can improve marketing campaigns, predict stock performance, and beat grand masters in Go and chess.
About the Book
Grokking Deep Reinforcement Learning uses engaging exercises to teach you how to build deep learning systems. This book combines annotated Python code with intuitive explanations to explore DRL techniques. You'll see how algorithms function and learn to develop your own DRL agents using evaluative feedback.
What's Inside
- An introduction to reinforcement learning
- DRL agents with human-like behaviors
- Applying DRL to complex situations
About the Reader
For developers with basic deep learning experience.
About the Author
Miguel Morales works on reinforcement learning at Lockheed Martin and is an instructor for the Georgia Institute of Technology’s Reinforcement Learning and Decision Making course.
Quotes
This book is very well put together. It explains in technical but clear language what machine learning is, what deep learning is, and what reinforcement learning is.
- From the Foreword by Charles Isbell
This is the best practical introduction to deep reinforcement learning you’ll find. Very highly recommended.
- Ike Okonkwo, Solid State AI
A fine introduction to deep reinforcement learning with just the right balance between math, concepts, and examples.
- Alain Couniot, Sopra Steria Benelux
Looking for a definitive source on deep reinforcement learning? Then look no further than this book. Highly recommended.
- Swaminathan Subramanian, DXC Technology
Table of contents
- Grokking Deep Reinforcement Learning
- Copyright
- dedication
- contents
- front matter
-
1 Introduction to deep reinforcement learning
-
What is deep reinforcement learning?
- Deep reinforcement learning is a machine learning approach to artificial intelligence
- Deep reinforcement learning is concerned with creating computer programs
- Deep reinforcement learning agents can solve problems that require intelligence
- Deep reinforcement learning agents improve their behavior through trial-and-error learning
- Deep reinforcement learning agents learn from sequential feedback
- Deep reinforcement learning agents learn from evaluative feedback
- Deep reinforcement learning agents learn from sampled feedback
- Deep reinforcement learning agents use powerful non-linear function approximation
- The past, present, and future of deep reinforcement learning
- The suitability of deep reinforcement learning
- Setting clear two-way expectations
- Summary
-
What is deep reinforcement learning?
-
2 Mathematical foundations of reinforcement learning
- Components of reinforcement learning
-
MDPs: The engine of the environment
- States: Specific configurations of the environment
- Actions: A mechanism to influence the environment
- Transition function: Consequences of agent actions
- Reward signal: Carrots and sticks
- Horizon: Time changes what’s optimal
- Discount: The future is uncertain, value it less
- Extensions to MDPs
- Putting it all together
- Summary
- 3 Balancing immediate and long-term goals
-
4 Balancing the gathering and use of information
-
The challenge of interpreting evaluative feedback
- Bandits: Single-state decision problems
- Regret: The cost of exploration
- Approaches to solving MAB environments
- Greedy: Always exploit
- Random: Always explore
- Epsilon-greedy: Almost always greedy and sometimes random
- Decaying epsilon-greedy: First maximize exploration, then exploitation
- Optimistic initialization: Start off believing it’s a wonderful world
- Strategic exploration
- Summary
-
The challenge of interpreting evaluative feedback
- 5 Evaluating agents’ behaviors
- 6 Improving agents’ behaviors
- 7 Achieving goals more effectively and efficiently
-
8 Introduction to value-based deep reinforcement learning
-
The kind of feedback deep reinforcement learning agents use
- Deep reinforcement learning agents deal with sequential feedback
- But, if it isn’t sequential, what is it?
- Deep reinforcement learning agents deal with evaluative feedback
- But, if it isn’t evaluative, what is it?
- Deep reinforcement learning agents deal with sampled feedback
- But, if it isn’t sampled, what is it?
- Introduction to function approximation for reinforcement learning
-
NFQ: The first attempt at value-based deep reinforcement learning
- First decision point: Selecting a value function to approximate
- Second decision point: Selecting a neural network architecture
- Third decision point: Selecting what to optimize
- Fourth decision point: Selecting the targets for policy evaluation
- Fifth decision point: Selecting an exploration strategy
- Sixth decision point: Selecting a loss function
- Seventh decision point: Selecting an optimization method
- Things that could (and do) go wrong
- Summary
-
The kind of feedback deep reinforcement learning agents use
- 9 More stable value-based methods
-
10 Sample-efficient value-based methods
-
Dueling DDQN: A reinforcement-learning-aware neural network architecture
- Reinforcement learning isn’t a supervised learning problem
- Nuances of value-based deep reinforcement learning methods
- Advantage of using advantages
- A reinforcement-learning-aware architecture
- Building a dueling network
- Reconstructing the action-value function
- Continuously updating the target network
- What does the dueling network bring to the table?
- PER: Prioritizing the replay of meaningful experiences
- Summary
-
Dueling DDQN: A reinforcement-learning-aware neural network architecture
- 11 Policy-gradient and actor-critic methods
- 12 Advanced actor-critic methods
- 13 Toward artificial general intelligence
- index
Product information
- Title: Grokking Deep Reinforcement Learning
- Author(s):
- Release date: December 2020
- Publisher(s): Manning Publications
- ISBN: 9781617295454
You might also like
book
Deep Reinforcement Learning Hands-On - Second Edition
New edition of the bestselling guide to deep reinforcement learning and how it's used to solve …
book
Generative Deep Learning, 2nd Edition
Generative AI is the hottest topic in tech. This practical book teaches machine learning engineers and …
book
Deep Reinforcement Learning with Python - Second Edition
An example-rich guide for beginners to start their reinforcement and deep reinforcement learning journey with state-of-the-art …
book
Deep Learning for Coders with fastai and PyTorch
Deep learning is often viewed as the exclusive domain of math PhDs and big tech companies. …