Hands-On Transfer Learning with Python

Write operation

Each write head receives an erase vector, e_t, and an add vector, a_t, to reset and write to memory, just like an LSTM cell, as follows: M_t(i) ← M_t(i) [1- e_{t ⁽ⁱ⁾} w_t (i) ] + w_t (i) a_t (i).

Here is the pseudo code for the preceding operations:

mem_size = 128 #The size of memory mem_dim = 16 #The dimensionality for memory shift_range = 1 # defining shift[-1, 0, 1]## last output layer from LSTM controller: last_output## Previous memory state: M_prevdef Linear(input_, output_size, stddev=0.5):  '''Applies a linear transformation to the input data: input_  implements dense layer with tf.random_normal_initializer(stddev=stddev)  as weight initializer  ''''def get_controller_head(M_prev, last_output, is_read=True):   k = tf.tanh(Linear(last_output, ...