Hands-On Mathematics for Deep Learning

Memory-augmented neural networks

As the name suggests, Memory-Augmented Neural Networks (MANNs) are augmented using external memory (a storage buffer), which makes it easier for the model to learn and retain new information so as to not forget it later on. One of the approaches that is used is training a Neural Turing Machine (NTM) to learn a learning algorithm by altering the training setup and memory retrieval.

To adapt an NTM for meta learning, we need it to be able to encode information related to new tasks quickly, while also ensuring that the stored information can be accessed quickly. The way this works is we pass the information at the present time step and the corresponding label at the next time step, which forces the network to ...