Hands-On Convolutional Neural Networks with TensorFlow

Substituting big convolutions

Before we jump in, we will first learn about the techniques that can reduce the number of parameters a model uses. This is important, firstly because it should improve your network's ability to generalize, as it will need less training data fed into it to utilize the number of parameters present in the model. Secondly, having less parameters means more hardware efficiency, as less memory will be needed.

Here, we will start by explaining one important technique for reducing model parameters, cascading several small convolutions together. In the diagram that follows, we have two 3x3 convolution layers. If we look at the second layer, on the right of the diagram, working back, we can see that one neuron in the ...