Meta Networks (MetaNet) is an architecture created to generalize across tasks quickly; it uses fast weights to do so. The reason they are called fast weights is that instead of using gradient descent to update weights as we normally do, we use a neural network to predict the weights for another neural network. The weights the other neural network generates are referred to as fast weights, while the weights that rely on gradient descent are referred to as slow weights. The effect of this is that it supports meta-level continual learning.
In the following figure, you can see the overall architecture of MetaNet:
MetaNet is comprised ...
