Hands-On Meta Learning with Python

Read operation

A read operation reads a value from the memory. But since we have many memory blocks in our memory matrix, which one do we need to select to read from the memory? That is determined by the weight vector. The weight vector specifies which region in the memory is more important than others. We use an attention mechanism to get this weight vector. We will explore more on how exactly we compute this weight vector in the upcoming section. The weight vector is normalized, meaning that its value ranges from zero to one, and the sum of the value equals one. The following diagram shows the weight vector of the length, N:

Let's denote ...