Math for Deep Learning

7DIFFERENTIAL CALCULUS

The discovery of “the calculus” by Sir Issac Newton, and separately by Gottfried Wilhelm Leibniz, was one of the greatest achievements in the history of mathematics. Calculus is typically split into two main parts: differential and integral. Differential calculus talks about rates of change and their relationships, embodied in the notion of the derivative. Integral calculus is concerned with things like the area under a curve.

We don’t need integral calculus for deep learning, but we’ll use differential calculus often. For example, we use differential calculus to train neural networks; we adjust the weights of a neural network ...