Practical Convolutional Neural Networks

ResNet architecture

After a certain depth, adding additional layers to feed-forward convNets results in a higher training error and higher validation error. When adding layers, performance increases only up to a certain depth, and then it rapidly decreases. In the ResNet (Residual Network) paper, the authors argued that this underfitting is unlikely due to the vanishing gradient problem, because this happens even when using the batch normalization technique. Therefore, they have added a new concept called residual block. The ResNet team added connections that can skip layers:

ResNet uses standard convNet and adds connections that skip a few convolution layers at a time. Each bypass gives a residual block.

Residual block

In the 2015 ImageNet ...