Advanced Deep Learning with Python

Training and evaluation

Next, let's implement the train function. It's similar to other such functions that we've implemented in previous chapters; however, it takes into account the sequential nature of the input and the teacher forcing principle we described in the Seq2eq with attention section. For the sake of simplicity, we'll only train with a single sequence at a time (a mini batch of size 1).

First, we'll initiate the iteration over the training set, set up initial sequence tensors, and reset the gradients:

def train(encoder, decoder, loss_function, encoder_optimizer, decoder_optimizer, data_loader, max_length=MAX_LENGTH):    print_loss_total = 0    # Iterate over the dataset    for i, (input_tensor, target_tensor) in enumerate(data_loader): ...