Deep Learning from the Basics

4. Neural Network Training

This chapter describes neural network training. When we talk about "training" in this context, we mean obtaining the optimal weight parameters automatically from training data. In this chapter, we will introduce a criterion called a loss function; this enables a neural network to learn. The purpose of training is to discover the weight parameters that lead to the smallest value of the loss function. In this chapter, we will be introduced to the method of using the gradient of a function, called a gradient method, to discover the smallest loss function value.

Learning from Data

The essential characteristic of a neural network is its ability to learn from data. Training from data means that weight parameter values ...