Reinforcement Learning Algorithms with Python

The experienced buffer

The experienced buffer is a class of the ExperienceBuffer type and stores a queue of type FIFO (First In, First Out) for each of the following components: observation, reward, action, next observation, and done. FIFO means that once it reaches the maximum capacity specified by maxlen, it discards the elements starting from the oldest one. In our implementation, the capacity is buffer_size:

class ExperienceBuffer():    def __init__(self, buffer_size):        self.obs_buf = deque(maxlen=buffer_size)        self.rew_buf = deque(maxlen=buffer_size)        self.act_buf = deque(maxlen=buffer_size)        self.obs2_buf = deque(maxlen=buffer_size)        self.done_buf = deque(maxlen=buffer_size)    def add(self, obs, rew, act, obs2, done):        self.obs_buf.append(obs) ...