Reinforcement Learning Algorithms with Python

FIM and KL divergence

The FIM is defined as the covariance of an objective function. Let's look at how it can help us. To be able to limit the distance between the distributions of our model, we need to define a metric that provides the distance between the new and the old distributions. The most popular choice is to use the KL divergence. It measures how far apart two distributions are and is used in many places in RL and machine learning. The KL divergence is not a proper metric as it is not symmetric, but it is a good approximation of it. The more different two distributions, are the higher the KL divergence value. Consider the plot in the following diagram. In this example, the KL divergences are computed with respect to the green function. ...