Book description
Start with the basics of reinforcement learning and explore deep learning concepts such as deep Q-learning, deep recurrent Q-networks, and policy-based methods with this practical guide
Key Features
- Use TensorFlow to write reinforcement learning agents for performing challenging tasks
- Learn how to solve finite Markov decision problems
- Train models to understand popular video games like Breakout
Book Description
Various intelligent applications such as video games, inventory management software, warehouse robots, and translation tools use reinforcement learning (RL) to make decisions and perform actions that maximize the probability of the desired outcome. This book will help you to get to grips with the techniques and the algorithms for implementing RL in your machine learning models.
Starting with an introduction to RL, you'll be guided through different RL environments and frameworks. You'll learn how to implement your own custom environments and use OpenAI baselines to run RL algorithms. Once you've explored classic RL techniques such as Dynamic Programming, Monte Carlo, and TD Learning, you'll understand when to apply the different deep learning methods in RL and advance to deep Q-learning. The book will even help you understand the different stages of machine-based problem-solving by using DARQN on a popular video game Breakout. Finally, you'll find out when to use a policy-based method to tackle an RL problem.
By the end of The Reinforcement Learning Workshop, you'll be equipped with the knowledge and skills needed to solve challenging problems using reinforcement learning.
What you will learn
- Use OpenAI Gym as a framework to implement RL environments
- Find out how to define and implement reward function
- Explore Markov chain, Markov decision process, and the Bellman equation
- Distinguish between Dynamic Programming, Monte Carlo, and Temporal Difference Learning
- Understand the multi-armed bandit problem and explore various strategies to solve it
- Build a deep Q model network for playing the video game Breakout
Who this book is for
If you are a data scientist, machine learning enthusiast, or a Python developer who wants to learn basic to advanced deep reinforcement learning algorithms, this workshop is for you. A basic understanding of the Python language is necessary.
Table of contents
- The Reinforcement Learning Workshop
-
Preface
-
About the Book
- Audience
- About the Chapters
- Conventions
- Code Presentation
- Setting up Your Environment
- Installing Anaconda for Jupyter Notebook
- Installing a Virtual Environment
- Installing Gym
- Installing TensorFlow 2
- Installing PyTorch
- Installing OpenAI Baselines
- Installing Pillow
- Installing Torch
- Installing Other Libraries
- Accessing the Code Files
-
About the Book
-
1. Introduction to Reinforcement Learning
- Introduction
- Learning Paradigms
- Fundamentals of Reinforcement Learning
- Reinforcement Learning Frameworks
- Applications of Reinforcement Learning
- Summary
-
2. Markov Decision Processes and Bellman Equations
- Introduction
- Markov Processes
-
3. Deep Learning in Practice with TensorFlow 2
- Introduction
- An Introduction to TensorFlow and Keras
-
How to Implement a Neural Network Using TensorFlow
- Model Creation
- Model Training
- Loss Function Definition
- Optimizer Choice
- Learning Rate Scheduling
- Feature Normalization
- Model Validation
- Performance Metrics
- Model Improvement
- Standard Fully Connected Neural Networks
- Exercise 3.02: Building a Fully Connected Neural Network Model with the Keras High-Level API
- Convolutional Neural Networks
- Exercise 3.03: Building a Convolutional Neural Network Model with the Keras High-Level API
- Recurrent Neural Networks
- Exercise 3.04: Building a Recurrent Neural Network Model with the Keras High-Level API
- Simple Regression Using TensorFlow
- Simple Classification Using TensorFlow
- TensorBoard – How to Visualize Data Using TensorBoard
- Summary
- 4. Getting Started with OpenAI and TensorFlow for Reinforcement Learning
- 5. Dynamic Programming
-
6. Monte Carlo Methods
- Introduction
- The Workings of Monte Carlo Methods
- Understanding Monte Carlo with Blackjack
-
Types of Monte Carlo Methods
- First Visit Monte Carlo Prediction for Estimating the Value Function
- Exercise 6.02: First Visit Monte Carlo Prediction for Estimating the Value Function in Blackjack
- Every Visit Monte Carlo Prediction for Estimating the Value Function
- Exercise 6.03: Every Visit Monte Carlo Prediction for Estimating the Value Function
- Exploration versus Exploitation Trade-Off
- Importance Sampling
- Solving Frozen Lake Using Monte Carlo
- Summary
-
7. Temporal Difference Learning
- Introduction to TD Learning
-
TD(0) – SARSA and Q-Learning
- SARSA – On-Policy Control
- Exercise 7.01: Using TD(0) SARSA to Solve FrozenLake-v0 Deterministic Transitions
- The Stochasticity Test
- Exercise 7.02: Using TD(0) SARSA to Solve FrozenLake-v0 Stochastic Transitions
- Q-Learning – Off-Policy Control
- Exercise 7.03: Using TD(0) Q-Learning to Solve FrozenLake-v0 Deterministic Transitions
- Expected SARSA
- N-Step TD and TD(λ) Algorithms
- The Relationship between DP, Monte-Carlo, and TD Learning
- Summary
- 8. The Multi-Armed Bandit Problem
-
9. What Is Deep Q-Learning?
- Introduction
- Basics of Deep Learning
- Basics of PyTorch
- The Action-Value Function (Q Value Function)
- Deep Q Learning
-
Challenges in DQN
- Correlation between Steps and the Convergence Issue
- Experience Replay
- The Challenge of a Non-Stationary Target
- The Concept of a Target Network
- Exercise 9.04: Implementing a Working DQN Network with Experience Replay and a Target Network in PyTorch
- The Challenge of Overestimation in a DQN
- Double Deep Q Network (DDQN)
- Activity 9.01: Implementing a Double Deep Q Network in PyTorch for the CartPole Environment
- Summary
- 10. Playing an Atari Game with Deep Recurrent Q-Networks
- 11. Policy-Based Methods for Reinforcement Learning
-
12. Evolutionary Strategies for RL
- Introduction
- Problems with Gradient-Based Methods
-
Introduction to Genetic Algorithms
- Exercise 12.02: Implementing Fixed-Value and Uniform Distribution Optimization Using GAs
- Components: Population Creation
- Exercise 12.03: Population Creation
- Components: Parent Selection
- Exercise 12.04: Implementing the Tournament and Roulette Wheel Techniques
- Components: Crossover Application
- Exercise 12.05: Crossover for a New Generation
- Components: Population Mutation
- Exercise 12.06: New Generation Development Using Mutation
- Application to Hyperparameter Selection
- Exercise 12.07: Implementing GA Hyperparameter Optimization for RNN Training
- NEAT and Other Formulations
- Exercise 12.08: XNOR Gate Functionality Using NEAT
- Activity 12.01: Cart-Pole Activity
- Summary
-
Appendix
- 1. Introduction to Reinforcement Learning
- 2. Markov Decision Processes and Bellman Equations
- 3. Deep Learning in Practice with TensorFlow 2
- 4. Getting started with OpenAI and TensorFlow for Reinforcement Learning
- 5. Dynamic Programming
- 6. Monte Carlo Methods
- 7. Temporal Difference Learning
- 8. The Multi-Armed Bandit Problem
- 9. What Is Deep Q-Learning?
- 10. Playing an Atari Game with Deep Recurrent Q-Networks
- 11. Policy-Based Methods for Reinforcement Learning
- 12. Evolutionary Strategies for RL
Product information
- Title: The Reinforcement Learning Workshop
- Author(s):
- Release date: August 2020
- Publisher(s): Packt Publishing
- ISBN: 9781800200456
You might also like
book
Reinforcement Learning
Reinforcement learning (RL) will deliver one of the biggest breakthroughs in AI over the next decade, …
video
Reinforcement Learning in Motion
We all learn by interacting with the world around us, constantly experimenting and interpreting the results. …
book
The Data Analysis Workshop
Learn how to analyze data using Python models with the help of real-world use cases and …
book
The Applied Artificial Intelligence Workshop
With knowledge and information shared by experts, take your first steps towards creating scalable AI algorithms …