Advanced Deep Learning with Python

1×1 convolutions

A 1×1 (or pointwise) convolution is a special case of convolution where each dimension of the convolution filter is of size 1 (1×1 in 2D convolutions and 1×1×1 in 3D). At first, this doesn't make sense—a 1×1 filter doesn't increase the receptive field size of the output neurons. The result of such a convolution would be pointwise scaling. But it can be useful in another way—we can use them to change the depth between the input and output volumes.

To understand this, let's recall that, in general, we have an input volume with a depth of D slices and M filters for M output slices. Each output slice is generated by applying a unique filter over all of the input slices. If we use a 1×1 filter and D != M, we'll have output slices ...