Appendix C
Information Utility Measures
To quantify the information gain provided by a sensor measurement,
it is necessary to define a measure of information utility. The intu-
ition we would like to exploit is that information content is inversely
related to the “size” of the high probability uncertainty region of the
estimate of x. In Section 5.3.2, we studied the Mahalanobis distance
and mutual information-based measures. This section will define
several other measures, as well as provide additional details on the
Mahalanobis and mutual information distance measures.
C.1 Covariance-Based Information Utility Measure
In the simplest case of a unimodal posterior distribution that can be
approximated by a Gaussian, we can derive utility measures based on
the covariance of the distribution p
X
(x). The determinant det()is
proportional to the volume of the rectangular region enclosing the
covariance ellipsoid. Hence, the information utility function for this
approximation can be chosen as
ˆ
φ( p
X
) =−det().
Although the volume of the high-probability region seems to be a
useful measure, there are cases in which this measure underestimates
the residual uncertainty. In case the smallest principal axis shrinks
to zero, the volume of the uncertainty ellipsoid is zero, while the
uncertainties along the remaining principal axes might remain large.
An alternative measure using only the covariance of a distribu-
tion p
X
(x) would be the trace tr(), which is proportional to the
313
314 Appendix C Information Utility Measures
circumference of the rectangular region enclosing the covariance
ellipsoid. Hence, the information utility function would be
ˆ
φ( p
X
) =−tr().
C.2 Fisher Information Matrix
Another measure of information is the Fisher information matrix,
F(x ), defined over a class of likelihood densities {p(z
N
1
|x)}
xS
, where
z
N
1
refers to the sequence z
1
, ..., z
N
and x takes values from space S.
The ij
th
component of F(x )is
F
ij
(x) =
p
z
N
1
x
x
i
ln p
z
N
1
x
x
j
ln p
z
N
1
x
dz
N
1
,
where x
i
is the i
th
component of x. The Cramer-Rao bound states
that the error covariance of any unbiased estimator of x satisfies
F
1
(x).
It can be shown that Fisher information is related to the surface area
of the high-probability region which is a notion of the “size” of the
region [47]. Similar to the covariance-based measures, possible forms
of the information utility function using the Fisher information are
ˆ
φ( p
X
) = det(F(x)),
quantifying the inverse of the volume of high-probability uncer-
tainty region, or
ˆ
φ( p
X
) = tr(F(x)).
However, calculation of the Fisher information matrix requires
explicit knowledge of the distribution. For the case when a Gaussian
C.3 Entropy of Estimation Uncertainty 315
distribution can approximate the posterior, the Fisher information
matrix is the inverse of the error covariance:
F =
1
.
If additionally the Markov assumption for consecutive estimation
steps holds, we can incrementally update the parameter estimate
using a Kalman filter for linear models. In this case, the Fisher
information matrix can be updated recursively and independent of
measurement values using the Kalman equations [164]:
F
(k)
= F
(k1)
+
H
(k)
T
R
(k)
1
H
(k)
, (C.1)
where H
(k)
and R
(k)
are the observation matrix, (2.2), and the
measurement noise covariance at estimation step k, respectively.
For nonlinear systems, a popular approach is to use the extended
Kalman filter, which is a linear estimator for nonlinear systems,
obtained by linearization of the nonlinear state and observation
equations. In this case, the information matrix F can be recursively
updated by
F
(k)
= F
(k1)
+
J
(k)
T
R
(k)
1
J
(k)
, (C.2)
where J is the Jacobian of the measurement model h(·) in (2.1). The
information gain can then be measured by the “size” of this informa-
tion matrix—for example, as the determinant or trace of F. Interested
readers are referred to [164] for details of an information filter version
of the Kalman filter.
C.3 Entropy of Estimation Uncertainty
If the distribution of the estimate is highly non-Gaussian (e.g., multi-
modal), then the covariance is a poor statistic of the uncertainty.
In this case, one possible utility measure is the information-theoretic

Get Wireless Sensor Networks now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.