November 2017
Intermediate to advanced
274 pages
6h 16m
English
Deep neural networks contain nonlinear hidden layers, and this makes them expressive models that can learn very complicated relationships between inputs and outputs. However, these complicated relationships will be the result of sampling noise. These complicated relationships might not exist in test data, leading to overfitting. Many techniques and methods have been developed to reduce this noise. These include stopping the training as soon as performance on a validation set starts getting worse, introducing weight penalties such as L1 and L2 regularization, and soft weight sharing (Nowlan and Hinton, 1992).
Read now
Unlock full access