Skip to Main Content
Google Hacks, 3rd Edition
book

Google Hacks, 3rd Edition

by Rael Dornfest, Paul Bausch, Tara Calishain
August 2006
Beginner to intermediate content levelBeginner to intermediate
546 pages
14h 42m
English
O'Reilly Media, Inc.
Content preview from Google Hacks, 3rd Edition

Chapter 8. Programming Google

When search engines first appeared on the scene, they were more open to being spidered, scraped, and aggregated. Sites such as Excite and AltaVista didn’t worry too much about the odd surfer using Perl to grab a slice of a page or meta-search engines including their results in aggregated search results. Sure, egregious data suckers might get shut out, but the search engines weren’t worried about sharing their information on a smaller scale.

Google never took that stance. Instead, it has regularly prohibited meta-search engines from using its content without a license, and it tries its best to block unidentified web agents such as Perl’s LWP::Simple module or even wget on the command line. Google has even been known to block IP address ranges for running automated queries.

Google had every right to do this; after all, it was its search technology, database, and computer power. Unfortunately, however, these policies meant that casual researchers and Google nuts, like you and I, couldn’t play with its rich dataset in any automated way.

Google changed all that with the release of the Google Web API (http://api.google.com) in the spring of 2002. The Google Web API doesn’t allow you to do every kind of search possible—for example, it doesn’t support the phonebook: syntax—but it does make available Google’s eight-billion-page web database so that developers can create their own interfaces and use Google search results to their liking.

Tip

API stands for “Application ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Google Hacks, 2nd Edition

Google Hacks, 2nd Edition

Tara Calishain, Rael Dornfest
Google Hacks

Google Hacks

Tara Calishain, Rael Dornfest

Publisher Resources

ISBN: 0596527063Supplemental ContentErrata Page