Yahoo! Hacks

Hacks 68–76: Introduction

As you’ve already seen, Yahoo! provides an impressive amount of information on its sites. And through the years, private hackers have found ways to use Yahoo! data in their scripts and programs. A search for Yahoo! in the Perl module repository CPAN (http://search.cpan.org) will turn up hundreds of modules written by independent programmers who wanted to automate some piece of Yahoo! for their own purposes. Most of these Perl modules— and any code in other languages—rely on screen scraping to fetch the data. That is, the program downloads a Yahoo! web page and picks through the HTML to find the interesting data. Screen scraping isn’t a reliable way to fetch data, because a single change to the HTML means a change to the code is necessary to keep it working.

In February 2005, Yahoo! opened up a much more reliable and developer friendly path to its data, giving outside developers, tinkerers, and Yahoo! customers programmatic access to some of the data available on Yahoo! sites. This means that if you want to integrate information you find on Yahoo! into your own applications or web sites, Yahoo! Web Services has opened the door for you.