© Nikhil Ketkar 2017

Nikhil Ketkar, Deep Learning with Python, https://doi.org/10.1007/978-1-4842-2766-4_9

9. Automatic Differentiation

Nikhil Ketkar

(1)Bangalore, Karnataka, India

In the chapter on Stochastic Gradient Descent, we treated the computation of gradients of the loss function $$ {\nabla}_{\mathrm{x}}L(x) $$ as a black box. In this chapter we open this black box and cover the theory and practice of automatic differentiation. Automatic differentiation is a mature technology that allows for the effortless and efficient computation of gradients of arbitrarily complicated loss functions. This is critical when it comes to minimizing loss functions of interest; at the heart ...

Get Deep Learning with Python: A Hands-on Introduction now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.