Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

G.10 DIFFERENTIATION

Let A be an arbitrary nonzero matrix, and x and v be column vectors with appropriate dimensions. The following results are useful when differentiating matrix and vector expressions with respect to a column vector:

(G.65)

(G.66)

(G.67)

(G.68)

Example G.4. Consider the following system of linear equations:

(G.69)

If square matrix has full rank r = N, then it has an inverse and the solution is x = A−1y. However, if is rectangular such that M>N, the system of equations is overdetermined (A is a tall narrow matrix if ). For this case, we seek to minimize the following LS cost function:

(G.70)

Differentiating with respect to x and setting the result equal to zero ...