Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd Edition

Chapter 6The second differential

1 INTRODUCTION

In this chapter, we discuss second‐order partial derivatives, twice differentiability, and the second differential. Special attention is given to the relationship between twice differentiability and second‐order approximation. We define the Hessian matrix (for real‐valued functions) and find conditions for its symmetry. We also obtain a chain rule for Hessian matrices, and its analog for second differentials. Taylor's theorem for real‐valued functions is proved. Finally, we briefly discuss higher‐order differentials and show how the calculus of vector functions can be extended to matrix functions.

2 SECOND‐ORDER PARTIAL DERIVATIVES

Consider a vector function f : S → ℝ^m defined on a set S in ℝⁿ with values in ℝ^m. Let f_i : S → ℝ(i = 1, … , m) be the ith component function of f, and assume that f_i has partial derivatives not only at an interior point c of S but also at each point of an open neighborhood of c. Then we can also consider their partial derivatives, i.e. we can consider the limit

(1)

where e_k is the kth elementary vector in Rⁿ. When this limit exists, it is called the (k, j)th second‐order partial derivative of f_i at c and is denoted by . Thus, is obtained by first partially differentiating f_i with respect to the jth variable, ...