Skip to Main Content
Spidering Hacks
book

Spidering Hacks

by Morbus Iff, Tara Calishain
October 2003
Beginner to intermediate content levelBeginner to intermediate
428 pages
11h 9m
English
O'Reilly Media, Inc.
Content preview from Spidering Hacks

Chapter 4. Gleaning Data from Databases

Hack 48. Hacks #43-89

In Chapter 3, you learned techniques for collecting media files. Now you’re going to take those lessons and move in a slightly different direction: gleaning data from databases and information collections.

Information collections can be as large and multifaceted as Google’s index of the World Wide Web, or as narrow and precise as King County health collections. You can scrape information as general as archives from Yahoo! Groups, or as targeted as game prices from GameStop.com. In this chapter, we’ll look at a variety of ways you can access database information, a variety of sources you might want to try (and a few hints for thinking of your own!), and ways that you can combine the power of programming with already-existing web APIs to make new and powerful tools.

Hack #43. Archiving Yahoo! Groups Messages with yahoo2mbox

Looking to keep a local archive of your favorite mailing list? With yahoo2mbox, you can import the final results into your favorite mailer.

With the popularity of Yahoo! Groups (http://groups.yahoo.com/) comes a problem. Sometimes, you want to save the archives of a Yahoo! Group, but you want to be able to access it outside the Yahoo! Groups site. Or you want to move your list somewhere else and be able to take your existing archive with you.

Vadim Zeitlin had these same concerns, which is why he wrote yahoo2mbox (http://www.lpthe.jussieu.fr/~zeitlin/yahoo2mbox.html). This hack retrieves all the messages ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

QuickBooks 2005: The Missing Manual

QuickBooks 2005: The Missing Manual

Bonnie Biafore

Publisher Resources

ISBN: 0596005776Supplemental Content