Deep Learning Cookbook

Chapter 10. Building an Inverse Image Search Service

In the previous chapter we saw how to use a pretrained network on our own images, first by running a classifier on top of the network and then in a more complex example in which we trained only part of a network to recognize new classes of images. In this chapter we will use a similar approach to build a reverse image search engine, or a search by example.

We’ll start by looking at how we can acquire a good base set of images from Wikipedia by querying Wikidata. We’ll then use a pretrained network to extract values for each of those images to get embeddings. Once we have those embeddings, finding similar images is a mere matter of nearest neighbor search. Finally, we’ll look at principal components analysis (PCA) as a way to visualize relationships between images.

The code for this chapter can be found in the following notebook:

10.1 Building an inverse image search service

10.1 Acquiring Images from Wikipedia

Problem

How do you get a clean set of images from Wikipedia covering the major categories of things?

Solution

Use Wikidata’s metainformation to find Wikipedia pages that represent a class of things.

Wikipedia contains a great number of pictures that can be used for most purposes. The vast majority of those pictures, though, represent concrete instances, which is not really what we need for a reverse search engine. We want to return a picture representative of a cat as a species, not a specific cat like Garfield.