Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition

Chapter 10Second‐order differentials and Hessian matrices

1 INTRODUCTION

While in Chapter 9 the main tool was the first identification theorem, in this chapter it is the second identification theorem (Theorem 6.6) which plays the central role. The second identification theorem tells us how to obtain the Hessian matrix from the second differential, and the purpose of this chapter is to demonstrate how this theorem can be applied in practice.

In applications, one typically needs the first derivative of scalar, vector, and matrix functions, but one only needs the second derivative of scalar functions. This is because, in practice, second‐order derivatives typically appear in optimization problems and these are always univariate.

Very occasionally one might need the Hessian matrix of a vector function f = (f_s) or of a matrix function F = (f_st). There is not really a good notation and there are no useful matrix results for such ‘super‐Hessians’. The best way to handle these problems is to simply write the result as Hf_s(X) or Hf_st(X), that is, for each component of f or each element of F separately, and this is what we shall do; see Sections 10.9–10.11.

2 THE SECOND IDENTIFICATION TABLE

For a scalar function ϕ of an n × 1 vector x, the Hessian matrix of ϕ at x was introduced in Section 6.3 — it is the n × n matrix of second‐order partial derivatives denoted by

We note that

(1)

The ...