Grokking Deep Learning

Chapter 13. Introducing automatic optimization: let’s build a deep learning framework

In this chapter

• What is a deep learning framework?
• Introduction to tensors
• Introduction to autograd
• How does addition backpropagation work?
• How to learn a framework
• Nonlinearity layers
• The embedding layer
• The cross-entropy layer
• The recurrent layer

“Whether we are based on carbon or on silicon makes no fundamental difference; we should each be treated with appropriate respect.”

Arthur C. Clarke, 2010: Odyssey Two (1982)

What is a deep learning framework?

Good tools reduce errors, speed development, and increase runtime performance

If you’ve been reading about deep learning for long, you’ve probably come across one of the major frameworks such as PyTorch, ...