Appendix C

Information Utility Measures

To quantify the information gain provided by a sensor measurement,

it is necessary to deﬁne a measure of information utility. The intu-

ition we would like to exploit is that information content is inversely

related to the “size” of the high probability uncertainty region of the

estimate of x. In Section 5.3.2, we studied the Mahalanobis distance

and mutual information-based measures. This section will deﬁne

several other measures, as well as provide additional details on the

Mahalanobis and mutual information distance measures.

C.1 Covariance-Based Information Utility Measure

In the simplest case of a unimodal posterior distribution that can be

approximated by a Gaussian, we can derive utility measures based on

the covariance of the distribution p

X

(x). The determinant det()is

proportional to the volume of the rectangular region enclosing the

covariance ellipsoid. Hence, the information utility function for this

approximation can be chosen as

ˆ

φ( p

X

) =−det().

Although the volume of the high-probability region seems to be a

useful measure, there are cases in which this measure underestimates

the residual uncertainty. In case the smallest principal axis shrinks

to zero, the volume of the uncertainty ellipsoid is zero, while the

uncertainties along the remaining principal axes might remain large.

An alternative measure using only the covariance of a distribu-

tion p

X

(x) would be the trace tr(), which is proportional to the

313

314 Appendix C Information Utility Measures

circumference of the rectangular region enclosing the covariance

ellipsoid. Hence, the information utility function would be

ˆ

φ( p

X

) =−tr().

C.2 Fisher Information Matrix

Another measure of information is the Fisher information matrix,

F(x ), deﬁned over a class of likelihood densities {p(z

N

1

|x)}

x∈S

, where

z

N

1

refers to the sequence z

1

, ..., z

N

and x takes values from space S.

The ij

th

component of F(x )is

F

ij

(x) =

p

z

N

1

x

∂

∂x

i

ln p

z

N

1

x

∂

∂x

j

ln p

z

N

1

x

dz

N

1

,

where x

i

is the i

th

component of x. The Cramer-Rao bound states

that the error covariance of any unbiased estimator of x satisﬁes

≥ F

−1

(x).

It can be shown that Fisher information is related to the surface area

of the high-probability region which is a notion of the “size” of the

region [47]. Similar to the covariance-based measures, possible forms

of the information utility function using the Fisher information are

ˆ

φ( p

X

) = det(F(x)),

quantifying the inverse of the volume of high-probability uncer-

tainty region, or

ˆ

φ( p

X

) = tr(F(x)).

However, calculation of the Fisher information matrix requires

explicit knowledge of the distribution. For the case when a Gaussian

C.3 Entropy of Estimation Uncertainty 315

distribution can approximate the posterior, the Fisher information

matrix is the inverse of the error covariance:

F =

−1

.

If additionally the Markov assumption for consecutive estimation

steps holds, we can incrementally update the parameter estimate

using a Kalman ﬁlter for linear models. In this case, the Fisher

information matrix can be updated recursively and independent of

measurement values using the Kalman equations [164]:

F

(k)

= F

(k−1)

+

H

(k)

T

R

(k)

−1

H

(k)

, (C.1)

where H

(k)

and R

(k)

are the observation matrix, (2.2), and the

measurement noise covariance at estimation step k, respectively.

For nonlinear systems, a popular approach is to use the extended

Kalman ﬁlter, which is a linear estimator for nonlinear systems,

obtained by linearization of the nonlinear state and observation

equations. In this case, the information matrix F can be recursively

updated by

F

(k)

= F

(k−1)

+

J

(k)

T

R

(k)

−1

J

(k)

, (C.2)

where J is the Jacobian of the measurement model h(·) in (2.1). The

information gain can then be measured by the “size” of this informa-

tion matrix—for example, as the determinant or trace of F. Interested

readers are referred to [164] for details of an information ﬁlter version

of the Kalman ﬁlter.

C.3 Entropy of Estimation Uncertainty

If the distribution of the estimate is highly non-Gaussian (e.g., multi-

modal), then the covariance is a poor statistic of the uncertainty.

In this case, one possible utility measure is the information-theoretic

Get *Wireless Sensor Networks* now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.