Webbots, Spiders, and Screen Scrapers, 2nd Edition

Further Exploration

You can expand this basic webbot to do a variety of very useful things. Here is a short list of ideas to get you started on advanced designs.

• Create a web page with a form that allows people to enter and test the links of any web page.

• Schedule a link-verification bot to run periodically to ensure that links on web pages remain current. (For information on scheduling webbots, read Chapter 22.)

• Modify the webbot to send email notifications when it finds dead links. (More information on webbots that send email is available in Chapter 15.)

• Encase the webbot in a spider to check the links on an entire website.

• Convert this webbot into a function that is called directly from PHP. (This idea is explored in Chapter 16.)

• Eventually, you ...