Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition
by Jan R. Magnus, Heinz Neudecker
Chapter 10Second‐order differentials and Hessian matrices
1 INTRODUCTION
While in Chapter 9 the main tool was the first identification theorem, in this chapter it is the second identification theorem (Theorem 6.6) which plays the central role. The second identification theorem tells us how to obtain the Hessian matrix from the second differential, and the purpose of this chapter is to demonstrate how this theorem can be applied in practice.
In applications, one typically needs the first derivative of scalar, vector, and matrix functions, but one only needs the second derivative of scalar functions. This is because, in practice, second‐order derivatives typically appear in optimization problems and these are always univariate.
Very occasionally one might need the Hessian matrix of a vector function f = (fs) or of a matrix function F = (fst). There is not really a good notation and there are no useful matrix results for such ‘super‐Hessians’. The best way to handle these problems is to simply write the result as Hfs(X) or Hfst(X), that is, for each component of f or each element of F separately, and this is what we shall do; see Sections 10.9–10.11.
2 THE SECOND IDENTIFICATION TABLE
For a scalar function ϕ of an n × 1 vector x, the Hessian matrix of ϕ at x was introduced in Section 6.3 — it is the n × n matrix of second‐order partial derivatives
denoted by
We note that
The ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access