Math for Deep Learning

8MATRIX CALCULUS

Chapter 7 introduced us to differential calculus. In this chapter, we’ll discuss matrix calculus, which extends differentiation to functions involving vectors and matrices.

Deep learning works extensively with vectors and matrices, so it makes sense to develop a notation and approach to representing derivatives involving these objects. That’s what matrix calculus gives us. We saw a hint of this at the end of Chapter 7, when we introduced the gradient to represent the derivative of a scalar function of a vector—a function that accepts a vector argument and returns a scalar, f(x).

We’ll start with the table of matrix calculus derivatives ...