Deep Learning from the Basics

6. Training Techniques

This chapter describes important ideas in neural network training, including the optimization techniques that are used to search for optimal weight parameters, the initial values of weight parameters, and the method for setting hyperparameters—all of which are important topics when it comes to neural network training. We will look at regularization methods such as weight decay and dropout to prevent overfitting and implement them. Lastly, we will look at batch normalization, which has been used in a lot of research in recent years. By using the methods described in this chapter, you will be able to promote neural network training efficiently to improve recognition accuracy.

Updating Parameters

The purpose of neural ...