Deep Reinforcement Learning in Action

A.2. Calculus

Calculus is essentially the study of differentiation and integration. In deep learning, we only really need to use differentiation. Differentiation is the process of getting a derivative of a function.

We already introduced one notion of derivative: the ratio of an output interval to the input interval. It tells you how much the output space is stretched or squished. Importantly, these intervals are oriented intervals so they can be negative or positive, and thus the ratio can be negative or positive.

For example, consider the function f(x) = x². Take a point x and its neighborhood (x – ε,x + ε), where ε is some arbitrarily small value, and we get an interval around x. To be concrete, let x = 3, ε = 0.1; the interval around x = ...