AI and ML for Coders in PyTorch

Chapter 3. Going Beyond the Basics: Detecting Features in Images

In Chapter 2, you learned how to get started with computer vision by creating a simple neural network that matched the input pixels of the Fashion MNIST dataset to 10 labels, each of which represented a type (or class) of clothing. And while you created a network that was pretty good at detecting clothing types, there was a clear drawback. Your neural network was trained on small monochrome images, each of which contained only a single item of clothing, and each item was centered within the image.

To take the model to the next level, you need it to be able to detect features in images. So, for example, instead of looking merely at the raw pixels in the image, what if we could filter the images down to constituent elements? Matching those elements, instead of raw pixels, would help the model detect the contents of images more effectively. For example, consider the Fashion MNIST dataset that we used in the last chapter. When detecting a shoe, the neural network may have been activated by lots of dark pixels clustered at the bottom of the image, which it would see as the sole of the shoe. But if the shoe were not centered and filling the frame, this logic wouldn’t hold.

One method of detecting features comes from photography and image processing methodologies that you may already be familiar with. If you’ve ever used a tool like Photoshop or GIMP to sharpen an image, you’ve used a mathematical filter that works on the ...