Iterative Optimization in Inverse Problems

Background 21

Notice that we can write x

k+1

j

as a weighted geometric mean of x

k

j

and

x

k

j



b

i

(Ax

k

)

i



:

x

k+1

j

=



x

k

j



1−m

−1

i

A

ij



x

k

j



b

i

(Ax

k

)

i



m

−1

i

A

ij

. (1.31)

This will help to motivate the EMART.

1.5.4.3 Cross-Entropy

For a>0andb>0, let the cross-entropy or Kullback-Leibler (KL)

distance [153] from a to b be

KL(a, b)=a log

a

b

+ b − a, (1.32)

with KL(a, 0) = +∞, and KL(0,b)=b. Extend to nonnegative vectors

coordinate-wise, so that

KL(x, z)=

J



j=1

KL(x

j

,z

j

). (1.33)

Then KL(x, z) ≥ 0andKL(x, z) = 0 if and only if x = z.

Unlike the Euclidean distance, the KL distance is not symmetric;

KL(Ax, b)andKL(b, Ax) are distinct, and we can obtain different ap-

proximate solutions