Reinforcement Learning for Finance

Chapter 1. Learning Through Interaction

The idea that we learn by interacting with our environment is probably the first to occur to us when we think about the nature of learning.

Sutton and Barto (2018)

For human beings and animals alike, learning is almost as fundamental as breathing. It is something that happens continuously and most often unconsciously. There are different forms of learning. The one most important to the topics covered in this book is based on interacting with an environment.

Interaction with an environment provides the learner—or agent henceforth—with feedback that can be used to update their knowledge or to refine a skill. In this book, we are mostly interested in learning quantifiable facts about an environment, such as the odds of winning a bet or the reward that an action yields.

The next section discusses Bayesian learning as an example of learning through interaction. “Reinforcement Learning” presents breakthroughs in AI that were made possible through RL. It also describes the major building blocks of RL. “Deep Q-Learning” explains the two major characteristics of DQL, which is the most important algorithm in the remainder of the book.

Bayesian Learning

Two examples illustrate learning by interacting with an environment: tossing a biased coin and rolling a biased die. The examples are based on the idea that an agent betting repeatedly on the outcome of a biased gamble (and remembering all outcomes) can learn bet-by-bet about a gamble’s bias and ...