Keras Reinforcement Learning Projects

Keras Reinforcement Learning Projects

by Giuseppe Ciaburro
Released September 2018
Publisher(s): Packt Publishing
ISBN: 9781789342093

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

A practical guide to mastering reinforcement learning algorithms using Keras

Key Features

  • Build projects across robotics, gaming, and finance fields, putting reinforcement learning (RL) into action
  • Get to grips with Keras and practice on real-world unstructured datasets
  • Uncover advanced deep learning algorithms such as Monte Carlo, Markov Decision, and Q-learning

Book Description

Reinforcement learning has evolved a lot in the last couple of years and proven to be a successful technique in building smart and intelligent AI networks. Keras Reinforcement Learning Projects installs human-level performance into your applications using algorithms and techniques of reinforcement learning, coupled with Keras, a faster experimental library.

The book begins with getting you up and running with the concepts of reinforcement learning using Keras. You'll learn how to simulate a random walk using Markov chains and select the best portfolio using dynamic programming (DP) and Python. You'll also explore projects such as forecasting stock prices using Monte Carlo methods, delivering vehicle routing application using Temporal Distance (TD) learning algorithms, and balancing a Rotating Mechanical System using Markov decision processes.

Once you've understood the basics, you'll move on to Modeling of a Segway, running a robot control system using deep reinforcement learning, and building a handwritten digit recognition model in Python using an image dataset. Finally, you'll excel in playing the board game Go with the help of Q-Learning and reinforcement learning algorithms.

By the end of this book, you'll not only have developed hands-on training on concepts, algorithms, and techniques of reinforcement learning but also be all set to explore the world of AI.

What you will learn

  • Practice the Markov decision process in prediction and betting evaluations
  • Implement Monte Carlo methods to forecast environment behaviors
  • Explore TD learning algorithms to manage warehouse operations
  • Construct a Deep Q-Network using Python and Keras to control robot movements
  • Apply reinforcement concepts to build a handwritten digit recognition model using an image dataset
  • Address a game theory problem using Q-Learning and OpenAI Gym

Who this book is for

Keras Reinforcement Learning Projects is for you if you are data scientist, machine learning developer, or AI engineer who wants to understand the fundamentals of reinforcement learning by developing practical projects. Sound knowledge of machine learning and basic familiarity with Keras is useful to get the most out of this book

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Keras Reinforcement Learning Projects
  3. Packt Upsell
    1. Why subscribe?
    2. Packt.com
  4. Contributors
    1. About the author
    2. About the reviewer
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  6. Overview of Keras Reinforcement Learning
    1. Basic concepts of machine learning
      1. Discovering the different types of machine learning
        1. Supervised learning
        2. Unsupervised learning
        3. Reinforcement learning
      2. Building machine learning models step by step
    2. Getting started with reinforcement learning
      1. Agent-environment interface
      2. Markov Decision Process
      3. Discounted cumulative reward
      4. Exploration versus exploitation
    3. Reinforcement learning algorithms
      1. Dynamic Programming
      2. Monte Carlo methods
      3. Temporal difference learning
        1. SARSA
        2. Q-learning
        3. Deep Q-learning
    4. Summary
  7. Simulating Random Walks
    1. Random walks
      1. One-dimensional random walk
      2. Simulating 1D random walk
    2. Markov chains
      1. Stochastic process
      2. Probability calculation
      3. Markov chain definition
        1. Transition matrix
        2. Transition diagram
      4. Weather forecasting with Markov chains
      5. Generating pseudorandom text with Markov chains
    3. Summary
  8. Optimal Portfolio Selection
    1. Dynamic Programming
      1. Divide and conquer versus Dynamic Programming
      2. Memoization
      3. Dynamic Programming in reinforcement-learning applications
    2. Optimizing a financial portfolio
      1. Optimization techniques
    3. Solving the knapsack problem using Dynamic Programming
      1. Different approaches to the problem
        1. Brute force
        2. Greedy algorithms
        3. Dynamic Programming
    4. Summary
  9. Forecasting Stock Market Prices
    1. Monte Carlo methods
      1. Historical background
      2. Basic concepts of the Monte Carlo simulation
        1. Monte Carlo applications
        2. Numerical integration using the Monte Carlo method
      3. Monte Carlo for prediction and control
    2. Amazon stock price prediction using Python
      1. Exploratory analysis
      2. The Geometric Brownian motion model
      3. Monte Carlo simulation
    3. Summary
  10. Delivery Vehicle Routing Application
    1. Temporal difference learning
      1. SARSA
      2. Q-learning
    2. Basics of graph theory
      1. The adjacency matrix
      2. Adjacency lists
      3. Graphs as data structures in Python
      4. Graphs using the NetworkX package
    3. Finding the shortest path
      1. The Dijkstra algorithm
        1. The Dijkstra algorithm using the NetworkX package
        2. The Google Maps algorithm
    4. The Vehicle Routing Problem
    5. Summary
  11. Continuous Balancing of a Rotating Mechanical System
    1. Neural network basic concepts
      1. The Keras neural network model
    2. Classifying breast cancer using the neural network
    3. Deep reinforcement learning
      1. The Keras–RL package
    4. Continuous control with deep reinforcement learning
    5. Summary
  12. Dynamic Modeling of a Segway as an Inverted Pendulum System
    1. How Segways work
      1. System modeling basics
    2. OpenAI Gym
      1. OpenAI Gym methods
      2. OpenAI Gym installation
    3. The CartPole system
      1. Q-learning solution
      2. Deep Q-learning solution
    4. Summary
  13. Robot Control System Using Deep Reinforcement Learning
    1. Robot control
      1. Robotics overview
      2. Robot evolution
        1. First-generation robots
        2. Second-generation robots
        3. Third-generation robots
        4. Fourth-generation robots
      3. Robot autonomy
      4. Robot mobility
      5. Automatic control
        1. Control architectures
    2. The FrozenLake environment
      1. The Q-learning solution
      2. A Deep Q-learning solution
    3. Summary
  14. Handwritten Digit Recognizer
    1. Handwritten digit recognition
      1. Optical Character Recognition
      2. Computer vision
    2. Handwritten digit recognition using an autoencoder
      1. Loading data
      2. Model architecture
    3. Deep autoencoder Q-learning
    4. Summary
  15. Playing the Board Game Go
    1. Game theory
      1. Basic concepts
      2. Game types
        1. Cooperative games
        2. Symmetrical games
        3. Zero-sum games
        4. Sequential games
      3. Game theory applications
        1. Prisoner's dilemma
        2. Stag hunt
        3. Chicken game
    2. The Go game
      1. Basic rules of the game
      2. Scoring rules
    3. The AlphaGo project
      1. The AlphaGo algorithm
        1. Monte Carlo Tree Search
        2. Convolutional networks
    4. Summary
  16. What's Next?
    1. Reinforcement-learning applications in real life
      1. DeepMind AlphaZero
      2. IBM Watson
      3. The Unity Machine Learning Agents toolkit
      4. FANUC industrial robots
      5. Automated trading systems using reinforcement learning
    2. Next steps for reinforcement learning
      1. Inverse reinforcement learning
      2. Learning by demonstration
      3. Deep Deterministic Policy Gradients
      4. Reinforcement learning from human preferences
      5. Hindsight Experience Replay
    3. Summary
  17. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Keras Reinforcement Learning Projects
  • Author(s): Giuseppe Ciaburro
  • Release date: September 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789342093