Signal theory gives us all the tools we need to properly understand the convolution operation: why it is so widely used in many different domains and why CNNs are so powerful. The convolution operation is used to study the response of certain physical systems when a signal is applied to their input. Different input stimuli can make a system, S, produce a different output, and the behavior of a system can be modeled using the convolution operation.
Let's start from the one-dimensional case by introducing the concept of the Linear Time-Invariant (LTI) system.
A system, S, that accepts an input signal and produces an output signal, , is an LTI system if the following properties hold:
- Linearity:
- Time invariance:
