O'Reilly logo

Webbots, Spiders, and Screen Scrapers by Michael Schrenk

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Further Exploration

The true power of webbots is that they can make decisions and take action with the information they find online. Here are a few suggestions for extending what you've learned to do with RSS or other data you choose to aggregate with your webbots.

  • Modify the script in Listing 12-8 to accept stories that don't contain a keyword.

  • Write an aggregation webbot that doesn't display information unless it finds it on two or more sources.

  • Design a webbot that looks for specific keywords in news stories and sends an email notification when those keywords appear.

  • Search blogs for spelling errors.

  • Find an RSS feed that posts scores from your favorite sports team. Parse and store the scores in a database for later statistical analysis.

  • Write a ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required