Book description
Start with the basics of reinforcement learning and explore deep learning concepts such as deep Qlearning, deep recurrent Qnetworks, and policybased methods with this practical guide
Key Features
 Use TensorFlow to write reinforcement learning agents for performing challenging tasks
 Learn how to solve finite Markov decision problems
 Train models to understand popular video games like Breakout
Book Description
Various intelligent applications such as video games, inventory management software, warehouse robots, and translation tools use reinforcement learning (RL) to make decisions and perform actions that maximize the probability of the desired outcome. This book will help you to get to grips with the techniques and the algorithms for implementing RL in your machine learning models.
Starting with an introduction to RL, you'll be guided through different RL environments and frameworks. You'll learn how to implement your own custom environments and use OpenAI baselines to run RL algorithms. Once you've explored classic RL techniques such as Dynamic Programming, Monte Carlo, and TD Learning, you'll understand when to apply the different deep learning methods in RL and advance to deep Qlearning. The book will even help you understand the different stages of machinebased problemsolving by using DARQN on a popular video game Breakout. Finally, you'll find out when to use a policybased method to tackle an RL problem.
By the end of The Reinforcement Learning Workshop, you'll be equipped with the knowledge and skills needed to solve challenging problems using reinforcement learning.
What you will learn
 Use OpenAI Gym as a framework to implement RL environments
 Find out how to define and implement reward function
 Explore Markov chain, Markov decision process, and the Bellman equation
 Distinguish between Dynamic Programming, Monte Carlo, and Temporal Difference Learning
 Understand the multiarmed bandit problem and explore various strategies to solve it
 Build a deep Q model network for playing the video game Breakout
Who this book is for
If you are a data scientist, machine learning enthusiast, or a Python developer who wants to learn basic to advanced deep reinforcement learning algorithms, this workshop is for you. A basic understanding of the Python language is necessary.
Publisher resources
Table of contents
 The Reinforcement Learning Workshop

Preface

About the Book
 Audience
 About the Chapters
 Conventions
 Code Presentation
 Setting up Your Environment
 Installing Anaconda for Jupyter Notebook
 Installing a Virtual Environment
 Installing Gym
 Installing TensorFlow 2
 Installing PyTorch
 Installing OpenAI Baselines
 Installing Pillow
 Installing Torch
 Installing Other Libraries
 Accessing the Code Files

About the Book

1. Introduction to Reinforcement Learning
 Introduction
 Learning Paradigms
 Fundamentals of Reinforcement Learning
 Reinforcement Learning Frameworks
 Applications of Reinforcement Learning
 Summary

2. Markov Decision Processes and Bellman Equations
 Introduction
 Markov Processes

3. Deep Learning in Practice with TensorFlow 2
 Introduction
 An Introduction to TensorFlow and Keras

How to Implement a Neural Network Using TensorFlow
 Model Creation
 Model Training
 Loss Function Definition
 Optimizer Choice
 Learning Rate Scheduling
 Feature Normalization
 Model Validation
 Performance Metrics
 Model Improvement
 Standard Fully Connected Neural Networks
 Exercise 3.02: Building a Fully Connected Neural Network Model with the Keras HighLevel API
 Convolutional Neural Networks
 Exercise 3.03: Building a Convolutional Neural Network Model with the Keras HighLevel API
 Recurrent Neural Networks
 Exercise 3.04: Building a Recurrent Neural Network Model with the Keras HighLevel API
 Simple Regression Using TensorFlow
 Simple Classification Using TensorFlow
 TensorBoard – How to Visualize Data Using TensorBoard
 Summary
 4. Getting Started with OpenAI and TensorFlow for Reinforcement Learning
 5. Dynamic Programming

6. Monte Carlo Methods
 Introduction
 The Workings of Monte Carlo Methods
 Understanding Monte Carlo with Blackjack

Types of Monte Carlo Methods
 First Visit Monte Carlo Prediction for Estimating the Value Function
 Exercise 6.02: First Visit Monte Carlo Prediction for Estimating the Value Function in Blackjack
 Every Visit Monte Carlo Prediction for Estimating the Value Function
 Exercise 6.03: Every Visit Monte Carlo Prediction for Estimating the Value Function
 Exploration versus Exploitation TradeOff
 Importance Sampling
 Solving Frozen Lake Using Monte Carlo
 Summary

7. Temporal Difference Learning
 Introduction to TD Learning

TD(0) – SARSA and QLearning
 SARSA – OnPolicy Control
 Exercise 7.01: Using TD(0) SARSA to Solve FrozenLakev0 Deterministic Transitions
 The Stochasticity Test
 Exercise 7.02: Using TD(0) SARSA to Solve FrozenLakev0 Stochastic Transitions
 QLearning – OffPolicy Control
 Exercise 7.03: Using TD(0) QLearning to Solve FrozenLakev0 Deterministic Transitions
 Expected SARSA
 NStep TD and TD(λ) Algorithms
 The Relationship between DP, MonteCarlo, and TD Learning
 Summary
 8. The MultiArmed Bandit Problem

9. What Is Deep QLearning?
 Introduction
 Basics of Deep Learning
 Basics of PyTorch
 The ActionValue Function (Q Value Function)
 Deep Q Learning

Challenges in DQN
 Correlation between Steps and the Convergence Issue
 Experience Replay
 The Challenge of a NonStationary Target
 The Concept of a Target Network
 Exercise 9.04: Implementing a Working DQN Network with Experience Replay and a Target Network in PyTorch
 The Challenge of Overestimation in a DQN
 Double Deep Q Network (DDQN)
 Activity 9.01: Implementing a Double Deep Q Network in PyTorch for the CartPole Environment
 Summary
 10. Playing an Atari Game with Deep Recurrent QNetworks
 11. PolicyBased Methods for Reinforcement Learning

12. Evolutionary Strategies for RL
 Introduction
 Problems with GradientBased Methods

Introduction to Genetic Algorithms
 Exercise 12.02: Implementing FixedValue and Uniform Distribution Optimization Using GAs
 Components: Population Creation
 Exercise 12.03: Population Creation
 Components: Parent Selection
 Exercise 12.04: Implementing the Tournament and Roulette Wheel Techniques
 Components: Crossover Application
 Exercise 12.05: Crossover for a New Generation
 Components: Population Mutation
 Exercise 12.06: New Generation Development Using Mutation
 Application to Hyperparameter Selection
 Exercise 12.07: Implementing GA Hyperparameter Optimization for RNN Training
 NEAT and Other Formulations
 Exercise 12.08: XNOR Gate Functionality Using NEAT
 Activity 12.01: CartPole Activity
 Summary

Appendix
 1. Introduction to Reinforcement Learning
 2. Markov Decision Processes and Bellman Equations
 3. Deep Learning in Practice with TensorFlow 2
 4. Getting started with OpenAI and TensorFlow for Reinforcement Learning
 5. Dynamic Programming
 6. Monte Carlo Methods
 7. Temporal Difference Learning
 8. The MultiArmed Bandit Problem
 9. What Is Deep QLearning?
 10. Playing an Atari Game with Deep Recurrent QNetworks
 11. PolicyBased Methods for Reinforcement Learning
 12. Evolutionary Strategies for RL
Product information
 Title: The Reinforcement Learning Workshop
 Author(s):
 Release date: August 2020
 Publisher(s): Packt Publishing
 ISBN: 9781800200456
You might also like
book
HandsOn Machine Learning with ScikitLearn, Keras, and TensorFlow, 2nd Edition
Through a series of recent breakthroughs, deep learning has boosted the entire field of machine learning. …
book
40 Algorithms Every Programmer Should Know
Learn algorithms for solving classic computer science problems with this concise guide covering everything from fundamental …
book
Data Science from Scratch, 2nd Edition
To really learn data science, you should not only master the tools—data science libraries, frameworks, modules, …
book
Grokking Deep Learning
Grokking Deep Learning teaches you to build deep learning neural networks from scratch! In his engaging …