Machine Learning for Finance

Markov processes and the bellman equation – A more formal introduction to RL

Following the long history of modern deep learning being a continuation of quantitative finance with more GPUs, the theoretical foundation of reinforcement learning lies in Markov models.

Note

Note: This section requires a bit of mathematical background knowledge. If you are struggling, there is a beautiful visual introduction by Victor Powell here: http://setosa.io/ev/markov-chains/.

A more formal, but still simple, introduction is available on the website Analytics Vidhya: https://www.analyticsvidhya.com/blog/2014/07/markov-chain-simplified/.

A Markov model describes a stochastic process with different states in which the probability of ending up in a specific state is ...