Grokking Deep Learning

Chapter 6. Building your first deep neural network: introduction to backpropagation

In this chapter

• The streetlight problem
• Matrices and the matrix relationship
• Full, batch, and stochastic gradient descent
• Neural networks learn correlation
• Overfitting
• Creating your own correlation
• Backpropagation: long-distance error attribution
• Linear versus nonlinear
• The secret to sometimes correlation
• Your first deep network
• Backpropagation in code: bringing it all together

“O Deep Thought computer,” he said, “the task we have designed you to perform is this. We want you to tell us...” he paused, “The Answer.”

Douglas Adams, The Hitchhiker’s Guide to the Galaxy

The streetlight problem

This toy problem considers how a network learns entire datasets ...