Advanced Deep Learning with Python

Implementing GRUs

In this section, we'll implement a GRU cell with PyTorch 1.3.1 by following the blueprint from the Implementing LSTM section. Let's get started:

1. First, we'll do the imports:

import mathimport torch

2. Next, we'll write the class definition and the init method. In LSTM, we were able to create a shared fully connected layer for all gates, because each gate required the same input combination of x_t and h_t-1. The GRU gates use different inputs, so we'll create separate fully connected operations for each GRU gate:

class GRUCell(torch.nn.Module):    def __init__(self, input_size: int, hidden_size: int):        """        :param input_size: input vector size        :param hidden_size: cell state vector size        """        super(GRUCell, self).__init__() self.input_size ...