Deep Learning with Python: A Hands-on Introduction

© Nikhil Ketkar 2017

Nikhil Ketkar, Deep Learning with Python, https://doi.org/10.1007/978-1-4842-2766-4_9

9. Automatic Differentiation

Nikhil Ketkar¹ 

(1)Bangalore, Karnataka, India

In the chapter on Stochastic Gradient Descent, we treated the computation of gradients of the loss function as a black box. In this chapter we open this black box and cover the theory and practice of automatic differentiation. Automatic differentiation is a mature technology that allows for the effortless and efficient computation of gradients of arbitrarily complicated loss functions. This is critical when it comes to minimizing loss functions of interest; at the heart ...