Pattern Recognition

It is interesting to note that the Kullback–Leibler divergence becomes symmetric if the family of distributions is further restricted to Gaussian distributions with equal variances, i.e., σ21 = σ22 = σ2, i.e., when p1(m| μ1,σ), p2(m| μ2,σ), one has that D(p1p2)) = D(p2p1). Figure 2.7 shows two pairs of Gaussian distributions with equal variance. The corresponding Kullback–Leibler divergence is noted in the diagram. In order to illustrate the asymmetric behavior of the Kullback–Leibler divergence, Figure 2.8 depicts a pair of Gaussian distributions ...