April 2020
Beginner to intermediate
156 pages
4h 47m
English
The memory matrix at time step t (Mt) has R rows and C columns. There's an attention mechanism that dictates which memory location the attention head should read from/write to. This attention vector, generated by the controller is a length-R vector, called the weight vector (wt), where each entry of this vector wt(i) is the weight for the ith row of the memory bank. The weight vector is normalized, which means it satisfies the following conditions:

Read now
Unlock full access