Data reader

The data reader script helps to generate trainable batches from the preprocessed training text from the data parser script. Let's start by importing the required methods:

import pickleimport random

This helper module helps generate trainable batches from the preprocessed training text.

class Data_Reader:    def __init__(self, cur_train_index=0, load_list=False):        self.training_data = pickle.load(open('data/conversations_lenmax22_formersents2_with_former', 'rb'))        self.data_size = len(self.training_data)        if load_list:            self.shuffle_list = pickle.load(open('data/shuffle_index_list', 'rb'))        else:                self.shuffle_list = self.shuffle_index()        self.train_index = cur_train_index

The following code gets the batch number from the data:

    def get_batch_num ...

Get Python Reinforcement Learning Projects now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.