
122 Modeling and Inverse Problems in the Presence of Uncertainty
FIGURE 4.3: Akaike informa tion criterio n: r elationship between
Kullback–Leibler information and the maximum value of the log-likelihood
function of a given approximating model.
4.2.1 Kullback–Leibler Information
Kullback–Leibler (K–L) information [29] is a well-known measure of “dis -
tance” betwe en two probability distributio n models. Hence, it can be used
to measure information lost when an approximating probability distribution
model is used to approximate the true probability distribution model. L et
p
0
denote the probability distribution model that actually generates the data