Reinforcement Learning for Finance

Part I. The Basics

The first part of the book covers the basics of reinforcement learning and provides background information. It consists of three chapters:

• Chapter 1 focuses on learning through interaction with four major examples: probability matching, Bayesian updating, reinforcement learning (RL), and deep Q-learning (DQL).

• Chapter 2 introduces concepts from dynamic programming (DP) and discusses DQL as an approach to approximate solutions to DP problems. The major theme is the derivation of optimal policies to maximize a given objective function through taking a sequence of actions and updating the optimal policy iteratively. DQL is illustrated based on the CartPole game from the Gymnasium Python package.

• Chapter 3 develops a first Finance environment that allows the DQL agent from Chapter 2 to learn a financial prediction game. Although the environment formally replicates the API of the CartPole, it misses some important characteristics that are needed to apply RL successfully.