Artificial Intelligence for Big Data

Dropout

Dropout is a popular regularization technique used to prevent overfitting. When the deep neural network memorizes all the training data due to the limited size of the samples and a network of right depth is utilized for training, it does not generalize well enough to produce accurate results with the new test data. This is termed overfitting. Dropout is used primarily for preventing overfitting. This is a simple technique to implement. During the training phase, the algorithm selects the nodes from the deep neural network to be dropped (activation value set to 0). With each epoch, a different set of nodes is selected based on a predefined probability. For example, if a dropout rate of 0.2 is selected, during each of the epochs, there ...