Webbots, Spiders, and Screen Scrapers, 2nd Edition

Creating the Image-Capturing Webbot

This example webbot relies on a library called LIB_download_images, which is available from this book’s website. This library contains the following functions:

• download_binary_file(), which safely downloads image files

• mkpath(), which makes directory structures on your hard drive

• download_images_for_page(), which downloads all the images on a page

For clarity, I will break down this library into highlights and accompanying explanations.

Figure 9-1. Re-creating a file structure for stored images

The first script (Example 9-2) shows the main webbot used in Example 9-1 and Figure 9-1.

Example 9-2. Executing the image-capturing ...