Python Deep Learning - Second Edition

Dropout

Dropout is a regularization technique, which can be applied to the output of some of the network layers. Dropout randomly and periodically removes some of the neurons (along with their input and output connections) from the network. During a training mini-batch, each neuron has a probability p to be stochastically dropped. This is to ensure that no neuron ends up relying too much on other neurons and "learns" something useful for the network instead. Dropout can be applied after convolutional, pooling, or fully-connected layers. In the following illustration, we can see a dropout for fully-connected layers:

An example of dropout on ...