Python Reinforcement Learning Projects

Python Reinforcement Learning Projects

by Sean Saito, Yang Wenzhuo, Rajalingappaa Shanmugamani
Released September 2018
Publisher(s): Packt Publishing
ISBN: 9781788991612

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Deploy autonomous agents in business systems using powerful Python libraries and sophisticated reinforcement learning models

Key Features

  • Implement Q-learning and Markov models with Python and OpenAI
  • Explore the power of TensorFlow to build self-learning models
  • Eight AI projects to gain confidence in building self-trained applications

Book Description

Reinforcement learning (RL) is the next big leap in the artificial intelligence domain, given that it is unsupervised, optimized, and fast. Python Reinforcement Learning Projects takes you through various aspects and methodologies of reinforcement learning, with the help of insightful projects.

You will learn about core concepts of reinforcement learning, such as Q-learning, Markov models, the Monte-Carlo process, and deep reinforcement learning. As you make your way through the book, you'll work on projects with various datasets, including numerical, text, video, and audio, and will gain experience in gaming, image rocessing, audio processing, and recommendation system projects. You'll explore TensorFlow and OpenAI Gym to implement a deep learning RL agent that can play an Atari game. In addition to this, you will learn how to tune and configure RL algorithms and parameters by building agents for different kinds of games. In the concluding chapters, you'll get to grips with building self-learning models that will not only uncover layers of data but also reason and make decisions.

By the end of this book, you will have created eight real-world projects that explore reinforcement learning and will have handson experience with real data and artificial intelligence (AI) problems.

What you will learn

  • Train and evaluate neural networks built using TensorFlow for RL
  • Use RL algorithms in Python and TensorFlow to solve CartPole balancing
  • Create deep reinforcement learning algorithms to play Atari games
  • Deploy RL algorithms using OpenAI Universe
  • Develop an agent to chat with humans
  • Implement basic actor-critic algorithms for continuous control
  • Apply advanced deep RL algorithms to games such as Minecraft
  • Autogenerate an image classifier using RL

Who this book is for

Python Reinforcement Learning Projects is for data analysts, data scientists, and machine learning professionals, who have working knowledge of machine learning techniques and are looking to build better performing, automated, and optimized deep learning models. Individuals who want to work on self-learning model projects will also find this book useful.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Python Reinforcement Learning Projects
  3. Packt Upsell
    1. Why subscribe?
    2. Packt.com
  4. Contributors
    1. About the authors
    2. About the reviewer
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  6. Up and Running with Reinforcement Learning
    1. Introduction to this book
      1. Expectations
      2. Hardware and software requirements
      3. Installing packages
    2. What is reinforcement learning?
      1. The agent
      2. Policy
      3. Value function
      4. Model
      5. Markov decision process (MDP)
    3. Deep learning
      1. Neural networks
      2. Backpropagation
      3. Convolutional neural networks
        1. Advantages of neural networks
      4. Implementing a convolutional neural network in TensorFlow
        1. TensorFlow
        2. The Fashion-MNIST dataset
        3. Building the network
        4. Methods for building the network
          1. build method
          2. fit method
    4. Summary
    5. References
  7. Balancing CartPole
    1. OpenAI Gym
      1. Gym
      2. Installation 
      3. Running an environment
      4. Atari
      5. Algorithmic tasks
      6. MuJoCo
      7. Robotics
    2. Markov models
      1. CartPole
    3. Summary
  8. Playing Atari Games
    1. Introduction to Atari games
    2. Building an Atari emulator
      1. Getting started
      2. Implementation of the Atari emulator
        1. Atari simulator using gym
    3. Data preparation
    4. Deep Q-learning
      1. Basic elements of reinforcement learning
      2. Demonstrating basic Q-learning algorithm
    5. Implementation of DQN
    6. Experiments
    7. Summary
  9. Simulating Control Tasks
    1. Introduction to control tasks
      1. Getting started
      2. The classic control tasks
    2. Deterministic policy gradient
      1. The theory behind policy gradient
      2. DPG algorithm
      3. Implementation of DDPG
      4. Experiments
    3. Trust region policy optimization
      1. Theory behind TRPO
      2. TRPO algorithm
      3. Experiments on MuJoCo tasks
    4. Summary
  10. Building Virtual Worlds in Minecraft
    1. Introduction to the Minecraft environment
    2. Data preparation
    3. Asynchronous advantage actor-critic algorithm
    4. Implementation of A3C
    5. Experiments
    6. Summary
  11. Learning to Play Go
    1. A brief introduction to Go
      1. Go and other board games
      2. Go and AI research
    2. Monte Carlo tree search
      1. Selection
      2. Expansion
      3. Simulation
      4. Update
    3. AlphaGo
      1. Supervised learning policy networks
      2. Reinforcement learning policy networks
      3. Value network
      4. Combining neural networks and MCTS
    4. AlphaGo Zero
      1. Training AlphaGo Zero
      2. Comparison with AlphaGo
    5. Implementing AlphaGo Zero
      1. Policy and value networks
        1. preprocessing.py
        2. features.py
        3. network.py
      2. Monte Carlo tree search
        1. mcts.py
      3. Combining PolicyValueNetwork and MCTS
        1. alphagozero_agent.py
      4. Putting everything together
        1. controller.py
        2. train.py
    6. Summary
    7. References
  12. Creating a Chatbot
    1. The background problem
      1. Dataset
    2. Step-by-step guide
      1. Data parser
      2. Data reader
      3. Helper methods
      4. Chatbot model
      5. Training the data
      6. Testing and results
    3. Summary
  13. Generating a Deep Learning Image Classifier
    1. Neural Architecture Search
      1. Generating and training child networks
      2. Training the Controller
      3. Training algorithm
    2. Implementing NAS
      1. child_network.py
      2. cifar10_processor.py
      3. controller.py
      4. Method for generating the Controller
        1. Generating a child network using the Controller
        2. train_controller method
        3. Testing ChildCNN
      5. config.py
      6. train.py
      7. Additional exercises
    3. Advantages of NAS
    4. Summary
  14. Predicting Future Stock Prices
    1. Background problem
    2. Data used
    3. Step-by-step guide
      1. Actor script
      2. Critic script
      3. Agent script
      4. Helper script
      5. Training the data
      6. Final result
    4. Summary
  15. Looking Ahead
    1. The shortcomings of reinforcement learning
      1. Resource efficiency
      2. Reproducibility
      3. Explainability/accountability
      4. Susceptibility to attacks
    2. Upcoming developments in reinforcement learning
      1. Addressing the limitations
      2. Transfer learning
      3. Multi-agent reinforcement learning
    3. Summary
    4. References
  16. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Python Reinforcement Learning Projects
  • Author(s): Sean Saito, Yang Wenzhuo, Rajalingappaa Shanmugamani
  • Release date: September 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781788991612