There are various types of convolution operations we can choose from, and different configurations in the same convolution layer lead to different results. Here, we will summarize the commonly used convolution operations and talk about their strengths and weaknesses:
- Vanilla convolution: This is the most common convolution operation in CNNs. A convolution takes fewer parameters than a fully connected layer (nn.Linear) with the same input/output size and can be calculated pretty fast with im2col (see Chapter 7, Image Restoration with GANs, for more details). You can use the following snippet to create a ReLu-Conv-BN group (of course, feel free to change the order of the three functions):
class ReLUConvBN(nn.Module): ...
