Deep Learning from Scratch

Chapter 3. Deep Learning from Scratch

You may not realize it, but you now have all the mathematical and conceptual foundations to answer the key questions about deep learning models that I posed at the beginning of the book: you understand how neural networks work—the computations involved with the matrix multiplications, the loss, and the partial derivatives with respect to that loss—as well as why those computations work (namely, the chain rule from calculus). We achieved this understanding by building neural networks from first principles, representing them as a series of “building blocks” where each building block was a single mathematical function. In this chapter, you’ll learn to represent these building blocks themselves as abstract Python classes and then use these classes to build deep learning models; by the end of this chapter, you will indeed have done “deep learning from scratch”!

We’ll also map the descriptions of neural networks in terms of these building blocks to more conventional descriptions of deep learning models that you may have heard before. For example, by the end of this chapter, you’ll know what it means for a deep learning model to have “multiple hidden layers.” This is really the essence of understanding a concept: being able to translate between high-level descriptions and low-level details of what is actually going on. Let’s begin building toward this translation. So far, we’ve described models just in terms of the operations that happen at a low ...