Before looking at how capsules work, let's try to revisit how neurons function. A neuron receives scalar inputs from the previous layer's neurons, multiplies them by the corresponding weights, and sums the outputs. This summed output is passed through some non-linearity (such as ReLU) that outputs a new scalar, which is passed on to next layer's neurons.
In contrast to this, capsules take a vector as an input, as well as output a vector. The following diagram illustrates the process of computing the output of a capsule:
Let's look at each step in detail:
- Capsule j (at the higher levels) receives the vector inputs from ...
