Machine and Deep Learning Using MATLAB

7 A Convolutional Neural Network (CNN) Architecture and Training

Convolutional neural networks (CNNs) are a popular type of deep learning tool commonly used for various applications, particularly with images as inputs. However, they are also applicable to other domains like text, signals, and continuous responses. CNNs are inspired by the organization of simple and complex cells in the biological cortex, which respond to specific subregions (receptive regions) of a visual field. Similarly, CNNs are designed to activate based on subregions of an image rather than the entire image. In contrast, the fully connected (FC) layer is responsible for processing the entire image region. This means that neurons in a CNN focus only on their designated subregions and do not handle spatial areas beyond their assigned jurisdiction in the image.

Notice that such subregions might overlap, hence the neurons of a CNN produce spatially correlated outcomes, unlike other types of neural networks, where neurons do not share any connections and produce independent outcomes. Hence, in an FC layer, the number of parameters (weights) can tremendously increase as the size of the input increases. However, in a typical CNN the number of parameters will be reduced as a result of reduced number of connections, shared weights, and down-sampling.

The architecture of a CNN can vary depending on the types and number of layers to include. The types and number of layers to include depend on the intended application ...