Python Deep Learning - Second Edition

1x1 convolutions

1x1 (pointwise) convolution is a special case of convolution, where each dimension of the convolution filter is of size 1 (1x1 in 2D convolutions and 1x1x1 in 3D). At first this doesn't make sense—a 1x1 filter doesn't increase the receptive field size of the output neurons. The result of such convolution would be just pointwise scaling. But it can be useful in another way - we can use them to change the depth between the input and output volumes. To understand this, let's recall that in the general case we have input volume with a depth of D slices and M filters for M output slices. Each output slice is generated by applying a unique filter over all input slices. If we use a 1x1 filter and D != M, we'll have output slices ...