Keras Deep Learning Cookbook

Sequential memory

Sequential memory is used by the DQN agent to store various states, actions, and rewards. It has the following data structures:

• observations (dict): Observations returned by the environment
• actions (int): Actions taken to obtain this observation
• rewards (float): Rewards obtained by taking this action
• terminals (Boolean): This is the state terminal:

In the code, these data structures are defined as shown in the following snippet:

self.actions = RingBuffer(limit)self.rewards = RingBuffer(limit)self.terminals = RingBuffer(limit)self.observations = RingBuffer(limit)