Hands-On Generative Adversarial Networks with Keras

Jensen-Shannon divergence

As we described in Chapter 2, Introduction to Generative Models, the JSD is a symmetric and smoothed version of the Kullback-Leibler divergence:

The implementation in Python is straightforward. First, we normally each distribution by dividing them by their respective norm such that the comparison is at the same scale. After normalizing the distributions, we compute the KL distance from P to M and Q to M, where M is the mean between the distributions P and Q:

from scipy.stats import entropyfrom numpy.linalg import normdef JSD(P, Q):    # Jensen-Shannon Divergence or Symmetric KL    _P = P / norm(P, ord=1)    _Q = Q / norm(Q, ...