AI and ML for Coders in PyTorch

Chapter 11. Using Convolutional and Recurrent Methods for Sequence Models

The last few chapters introduced you to sequence data. You saw how to predict it, first by using statistical methods and then by using basic ML methods with a deep neural network. You also explored how to tune the model’s hyperparameters for better performance.

In this chapter, you’ll look at additional techniques that may further enhance your ability to predict sequence data by using convolutional neural networks as well as recurrent neural networks.

Convolutions for Sequence Data

In Chapter 3, you were introduced to convolutions in which a two-dimensional (2D) filter was passed over an image to modify it and potentially extract features. Over time, the neural network learned which filter values were effective at matching the modifications that had been made to the pixels to their labels, thus effectively extracting features from the image. The same technique can be applied to numeric time series data, but with one modification: the convolution will be one dimensional (1D) instead of two dimensional.

Consider, for example, the series of numbers in Figure 11-1.

Figure 11-1. A sequence of numbers

A 1D convolution could operate on these as follows. Consider the convolution to be a 1 × 3 filter with filter values of –0.5, 1, and –0.5, respectively. In this case, the first value in the sequence will be lost ...