Creating the Image-Capturing Webbot

This example webbot relies on a library called LIB_download_images, which is available from this book’s website. This library contains the following functions:

  • download_binary_file(), which safely downloads image files

  • mkpath(), which makes directory structures on your hard drive

  • download_images_for_page(), which downloads all the images on a page

For clarity, I will break down this library into highlights and accompanying explanations.

Re-creating a file structure for stored images

Figure 9-1. Re-creating a file structure for stored images

The first script (Example 9-2) shows the main webbot used in Example 9-1 and Figure 9-1.

Example 9-2. Executing the image-capturing ...

Get Webbots, Spiders, and Screen Scrapers, 2nd Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.