Yahoo! Hacks | O'Reilly Media

BUY THIS BOOK

Print Book $24.95

Safari Books Online

What is this?

Print Book £17.50

What is this?

Looking to Reprint this content?

Tell a friend

Yahoo! Hacks Tips & Tools for Living on the Web Frontier

By Paul Bausch
Price: $24.95 USD
£17.50 GBP

Cover | Table of Contents | Colophon

Chapter 1: Search

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Many of us use search engines in the same way we we use street signs. We use them to navigate, to get our bearings, and to pinpoint our destination. We rarely stop to consider the signs themselves or look for more information they might be telling us. As with street signs, we'd be lost without search engines, and by taking a few minutes to contemplate the Yahoo! Web Search results page, you might find new ways to reach your destination.

Take a look at Figure 1-1, which shows a Yahoo! Web Search results page for the query ancient greece.

Figure 1-1: Yahoo! Web Search results

You can see the familiar numbered listing of search results, but there are a number of other bits of information on the page. Here's a look at what's available on a Yahoo! Search results page:

Navigation bar: You'll find the gray navigation bar at the top of the page on many pages at Yahoo! sites. The bar provides a consistent way to get to the main Yahoo! page (http://www.yahoo.com), the My Yahoo! portal [Hack #34] , and Yahoo! Mail [Hack #52] . The bar also indicates your login status by displaying your Yahoo! ID [Hack #3] or Guest, along with links that let you sign in to Yahoo! or sign out. You can also click the Help link at the far right of the navigation bar to read documentation about the site.
Search links: Just above the search form, you'll find links to other Yahoo! searches, including Images, Video [Hack #11] , the Yahoo! Directory [Hack #20] , Yahoo! News [Hack #32] , and Yahoo! Shopping [Hack #47] . You can click any of these search links to search with the exact query you used at one of the other Yahoo! Search properties so that you don't have to retype your search term.
Result count: Search results are returned as a number of pages, and you'll find your position within the results in the shaded blue bar. For instance, the first page of results will be labeled as Results 1–10, the second page will show Results 11–20, and so on. (You can adjust the number of results per page by setting your search preferences

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Hacks 1–22: Introduction

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Take a look at Figure 1-1, which shows a Yahoo! Web Search results page for the query ancient greece.

Figure 1-1: Yahoo! Web Search results

You can see the familiar numbered listing of search results, but there are a number of other bits of information on the page. Here's a look at what's available on a Yahoo! Search results page:

Navigation bar: You'll find the gray navigation bar at the top of the page on many pages at Yahoo! sites. The bar provides a consistent way to get to the main Yahoo! page (http://www.yahoo.com), the My Yahoo! portal [Hack #34] , and Yahoo! Mail [Hack #52] . The bar also indicates your login status by displaying your Yahoo! ID [Hack #3] or Guest, along with links that let you sign in to Yahoo! or sign out. You can also click the Help link at the far right of the navigation bar to read documentation about the site.
Search links: Just above the search form, you'll find links to other Yahoo! searches, including Images, Video [Hack #11] , the Yahoo! Directory [Hack #20] , Yahoo! News [Hack #32] , and Yahoo! Shopping [Hack #47] . You can click any of these search links to search with the exact query you used at one of the other Yahoo! Search properties so that you don't have to retype your search term.
Result count: Search results are returned as a number of pages, and you'll find your position within the results in the shaded blue bar. For instance, the first page of results will be labeled as Results 1–10, the second page will show Results 11–20, and so on. (You can adjust the number of results per page by setting your search preferences

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Fine-Tune Yahoo! Web Search Queries

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

By understanding how to phrase your searches, you'll find more relevant search results.

Using Yahoo! Web Search (http://search.yahoo.com) is deceptively simple. You can type in any word or phrase and find matches in documents across the Web. The trade-off for this simplicity is having to look through hundreds, thousands, or millions of results to find the documents that are actually useful to you. By understanding how Yahoo! expects queries to be phrased, you can limit the results to include only those documents most relevant to you—saving you the time of looking through extraneous results.

To start building sophisticated queries, you need to know the basics. The following search basics will help you refine your Yahoo! searches:

Keyword: By default, Yahoo! searches for all of the words you type into a search form. If you type grammar into the search form, Yahoo! will return documents that contain the word grammar. A search for grammar school will return documents that contain both words somewhere within the document, but not necessarily together.
Complete phrase: To search for words in a specific order, enclose the words in quotation marks. A search for "grammar school" will return documents that contain the complete phrase grammar school. You can combine keyword and phrase searches. To find documents that contain the phrase grammar school and also have the word Oregon somewhere in the document, you could search for "grammar school" Oregon.
OR keyword: You can change the default behavior of keyword searches by using the capitalized keyword OR between words. A search for grammar OR primary will return documents that contain either grammar or primary, but not necessarily both words.
Exclude words or phrases: To find documents without a certain word, you can use the minus sign (-) along with the word you want to exclude. If a search for Oregon school returns too many pages for schools in the city of Portland, you could type Oregon school -Portland to exclude any pages with the word Portland from the results.

Once you have the basics down, you can start mixing and matching, and grouping queries together with parentheses. To find documents that contain the word

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Save Time with Search Shortcuts

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

By using a few specific keywords, you can have Yahoo! answer common questions within your search results, provide specific information, or point directly to your answer.

At its most basic, Yahoo! Search allows you to type in search terms, click the Search button, and receive documents that include that word or phrase from across the Web. This makes Yahoo! a keyword search engine, and knowing this can help you put together your queries.

Yahoo! Search accepts a number of key phrases that can provide quick answers to common questions. To illustrate, here's an example in which search shortcuts can give you a faster answer than keywords alone.

Imagine you're in California and you'd like to call a friend in London, England. You can't remember the time difference and you don't want to call at 3 a.m. London time, so you turn to Yahoo! for help. Browsing to http://search.yahoo.com, you find yourself in front of the search form, about to type. What's the best query? If you were asking a human being for the answer you might be tempted to type in a complete question: what is the time in London?.

Because Yahoo! Search looks for matching words or phrases, you've told Yahoo! to find documents that include the words in your phrase. So the top results will likely be the web site for the London newspaper The Times and other documents that contain the search terms London and time. You'll probably find your answer at sites that are in the results, but there's a much faster way to tell Yahoo! Search exactly what you're looking for. Understanding the time zone and other search shortcuts will give you quick answers to some common questions.

When Yahoo! Search encounters a key phrase, Yahoo! tries to provide a direct answer, in addition to matching documents that include the terms in the phrase. For example, if you want to know the current time in London, you can type time in London and you'll find the current time in London above the search results, as shown in Figure 1-3.

Figure 1-3: An answer using a Yahoo! Search shortcut

By using this shortcut, you'll have the answer without looking through the results or visiting other sites. Replace

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Create a Yahoo! ID

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The key to many of the services at Yahoo! is a unique Yahoo! ID, and understanding how IDs work is key to keeping your Yahoo! ID private.

A Yahoo! ID is what distinguishes you from the crowd and what lets Yahoo! remember you across browsing sessions and across computers. If you've been using the Web for any amount of time, chances are good that you already have a Yahoo! ID. In November 2004, Nielsen/Netratings estimated that 55 million people around the world use Yahoo! Mail, and all of them sign in with a Yahoo! ID.

If you don't already have a Yahoo! ID, you can create one in less than five minutes. Browse to http://login.yahoo.com and click Sign Up Now for the new account form.

The most important decision to make as you fill out the new account form is what your Yahoo! ID will be. You'll use your ID anytime you want to access your personalized Yahoo! data, and your ID will determine what your Yahoo! Mail email address will be. If your Yahoo! ID is j0d00d, your email address is j0d00d@yahoo.com. The hardest part is finding an ID that is different from the 55 million Yahoo! IDs that already exist. The next hardest part is keeping the ID short and memorable. Keep in mind that you'll use this ID anytime you want to log in to Yahoo!. A short ID will save your fingers some work if you plan to log in to Yahoo! from your cell phone. And friends might want to contact you with Yahoo! Instant Messenger via your Yahoo! ID, so a short ID that is similar to your name will help them remember it.

Before filling out the form, go straight to the Yahoo! ID field and start trying potential IDs. Click the Check Availability of This ID button until you find something that's not taken. If you can get something with your initials or first and last name, consider yourself lucky.

When it's time to choose your password, it's tempting to recycle a password that you use in other places. Your Yahoo! ID and password is the only thing protecting your email from unauthorized readers, so it's a good idea to make the password unique and somewhat complex. Yahoo! requires at least six characters, but you can do better than that. A complex password should include upper-and lowercase letters, as well as symbols or numbers. A good trick for creating long, memorable passwords that are hard for hackers to guess is to think of the first line of your favorite poem or song and choose the first letter of each word as your password. So "Mary had a little lamb, its fleece was white as snow" would translate into the password

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Set Persistent Yahoo! Search Preferences

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Tweak your Yahoo! preferences to get the most out of your searching.

If you already have a Yahoo! ID, you can set some preferences that will affect the appearance and content of your Yahoo! searches. To set your preferences, first make sure you're logged in to Yahoo! by visiting http://login.yahoo.com and entering your Yahoo! ID and password. From there, browse to http://www.yahoo.com or http://search.yahoo.com and look for the Preferences link to the right of the search form, like the one highlighted in Figure 1-7.

Figure 1-7: Yahoo! Search Preferences link

If you don't see the preferences link, you can browse directly to http://search.yahoo.com/preferences. From the Preferences page, you can set a number of options that Yahoo! will remember and apply to any search results in the future.

When searching a particular topic, it's easy to click on a search result, get lost in reading, and find yourself several clicks away from your original page of search results. If you find yourself clicking your browser's Back button again and again to get back to your Yahoo! search results, you might want to open links from the search results page in a new browser window. You can set this preference by checking the New Window box on the Preferences page. This is handy for keeping your search results page in place, allowing you to browse other sites without fear of losing your search results.

By default, Yahoo! shows 20 results on each page. You can change this setting on the Preferences page to 10, 15, 20, 30, 40, or 100 results. Setting it to display fewer results per page will keep your scrolling to a minimum, but if you have a large screen you might appreciate seeing up to a hundred results without clicking through several pages.

As in other media, the Web is filled with material that isn't appropriate for children or the workplace. Yahoo! indexes the entire Web—including the seamier sections—and Yahoo!'s answer to this dilemma is the SafeSearch Filter. With SafeSearch enabled, Yahoo! will do its best to exclude any adult material from search results. There are three different settings you can apply to your Yahoo! ID:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Assemble Advanced Search Queries

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

By understanding how Yahoo! Advanced Search URLs are structured, you can create your own Advanced Search queries on the fly.

In addition to the simple search form you'll find at http://search.yahoo.com, Yahoo! offers an Advanced Web Search form at http://search.yahoo.com/web/advanced. This form lets you refine your search in a number of ways, so you can narrow the results to a more useful list.

For example, if you'd like to find information about a generic topic, such as astronomy, you could go to Yahoo!, type astronomy into the search form, and find hundreds of sites related to the word. But if you want only a segment of those results, you can browse over to the Advanced Web Search form, type astronomy, and limit the results by top-level domain, as shown in Figure 1-9.

Figure 1-9: Yahoo! Advanced Search form

A search for astronomy across .gov sites returns only pages at NASA's web site. The same search limited to .edu sites results in astronomy programs at various universities, and limiting to .com gives you astronomy magazines at the top of the results.

You can further refine your search by limiting it to a specific file format, such as PDF files, Excel spreadsheets, or XML files. For any given search, you can also override your global preferences settings for language, number of results, and adult content filtering.

To get started with hacking URLs, type a term into the Advanced Web Search form and click the Yahoo! Search button, which will take you to the results page. Once there, note the insanely long URL in the address of your browser. It will look something like this:

	http://search.yahoo.com/search?_adv_prop=web&x=op&ei=UTF-8&va=astronomy&va_
	vt=any&vp_vt=any&vo_vt=any&ve_vt=any&vd=all&vst=.gov&vs=.gov&vf=all&vm=p&  
	fl=0&n=20

For any given search URL, some of the variables you'll find in the URL are redundant or not necessary. The web form basically acts as a URL-building tool that has assembled this URL for you, and it isn't picky about which variables it includes. By understanding the pieces of the URL, you can construct your own queries using shorter URLs without the form.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Translate Any Page with Yahoo!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The World Wide Web has pages in every language, and Yahoo! can help you break through the language barrier.

Because the Web is a global space, we've all come across pages in different languages, especially among search results. If you're searching for information about a phrase like hamburger recipe, it's strange to come across a page about it in German. It's stranger still to find a mention of your name on a page in a foreign language. Imagine my surprise when I was searching Yahoo! for my name and found it at the Russian site shown in Figure 1-10.

Figure 1-10: Russian text with my name (Paul Bausch)

I can't read Russian, so of course I had no idea what the text said. I had recently added a photo gallery of old radio dials to my web site; I could tell they were linking to it, but I wanted to know what they were saying.

Yahoo!'s Language Tools page (http://tools.search.yahoo.com/language) has some ways to help you work with other languages. Among them is a translation service that will translate any block of text to a different language. I copied the Russian text from Figure 1-10, pasted it into the text area labeled "Translate this web text," chose "From Russian to English" from the drop-down list of languages, and clicked Translate. Yahoo! responded with this:

Radio Dials. The gallery of the photographs of ancient, I will not be
afraid this word, radios-scale. The author of collection, photographer
Paul Bausch, decided thus to publish the paternal collection of radio
receivers. 3x, dreams about their own tsifrozerkalke with the
macro-objective become increasingly more importunately.

As you can see, the Yahoo! translation tool isn't perfect, but it's good enough to give a sense of what the page is talking about. The translated text refers to the "paternal collection" of photos, because the radios I photographed belonged to my father. I still have no idea what the last sentence of the translation means, but I'm closer to understanding now than when it was in Russian.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Personalize, Track, and Share the Web

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

A free Yahoo! service called My Web remembers your searches, saves copies of web pages, and shares your saved pages with others.

The Yahoo! My Web beta gives you a powerful set of tools to collect web pages, annotate them, and share them with others. In My Web, you can save links to your favorite web sites (much like bookmarks) and organize them in custom folders. But there are several features available at My Web beyond collecting links:

Site Notes: You can add notes to any saved web site in My Web. You can use these notes to provide a site description, personal comments about the site, or any other bits of text.
Copies of Web Pages: Instead of saving just a link to a web site, My Web saves a copy of a page as it looked when you added it My Web. So even if a web page changes between when you added the site to My Web and when you want to reference it later, you can be sure you'll see the original information.
Search History: If you enable Search History, Yahoo! will remember which sites you click on in any Yahoo! search results and save them to your My Web History folder. If you have trouble remembering what search term you used to find a particular site, this might be a useful feature for you.
Blocked Sites: Some sites that show up in search results simply aren't relevant and won't be relevant to any search you make. Clicking Block tells Yahoo! not to show that particular site in your future search results. Blocked sites show up in your My Web Blocked folder, so you can periodically review the sites you've blocked.
Shared Folders: By default, folders with links and copied pages are private, but you can also choose to share any particular folder with the world. My Web also makes RSS feeds available for shared folders so others can subscribe to them and keep up with your changes.

To get started with My Web you just need a free Yahoo! ID. Browse to http://myweb.search.yahoo.com/myweb and log in with your Yahoo! ID. As you activate My Web, you'll have the option to import any existing Yahoo! Bookmarks (http://bookmarks.yahoo.com) and the option to download the latest Yahoo! Toolbar (

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Prefetch Yahoo! Search Results

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Automatically prefetch and cache the first search result on Yahoo! Web Search.

If you know how to use them properly, search engines are pretty darn good at finding exactly the page you're looking for. Google is so confident in its algorithm that it includes a hidden attribute in the search results page that tells Firefox to prefetch the first search result and cache it. You're probably going to click on the first result anyway, and when you do, it will load almost instantaneously because your browser has already been there.

Yahoo! Web Search is pretty good too, but it doesn't yet have this particular feature. So let's add it.

This hack relies on the Greasemonkey extension and thus works only in Firefox. If you're interested in doing much more with Greasemonkey, see Mark Pilgrim's forthcoming Greasemonkey Hacks, from which this hack is excerpted.

To begin, you'll need to install the Greasemonkey plug-in for Firefox. If you don't already have it, browse to http://greasemonkey.mozdev.org and click the Install Greasemonkey link. Follow the Software Installation prompts and then restart your browser. You'll know the plug-in is working if you see a small monkey icon in the lower-right corner of Firefox. Once installed, you can move on to analyzing Yahoo! and building the Greasemonkey script.

There are two important things about Yahoo! Search results that you can discover by viewing source on the search results page. First, the links of the search results each have a class yschttl. Yahoo uses this for styling the links with CSS, but you can use it to find the links in the first place. A single XPath query can extract a list of all the links with the class yschttl, and the first one of those is the one we want to prefetch and cache.

The second thing you need to know is that the search results Yahoo! provides are actually redirects through a tracking script on rds.yahoo.com that records which link you clicked on. A sample link looks like this:

	http://rds.yahoo.com/S=2766679/K=gpl+compatible/v=2/SID=e/TID=F510_112/  
	l=WS1/R=2/IPC=us/SHE=0/H=1/SIG=11sgv1lum/EXP=1116517280/*-http%3A//

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Compare Yahoo! and Google Search Results

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Pit Yahoo! and Google against each other and find more search results in the process.

If you've ever searched for the same phrase at both Yahoo! and Google, you've probably noticed that the results can be surprisingly different. That's because Yahoo! and Google have different ways of determining which sites are relevant for a particular phrase. Though both companies keep the exact way that they determine the rank of results a secret—to thwart people who would take advantage of it—both Yahoo! and Google provide some clues about what goes into their respective ranking systems.

Here's the official word from Yahoo!:

Yahoo! Search ranks results according to their relevance to a particular query by analyzing the web page text, title, and description accuracy as well as its source, associated links, and other unique document characteristics.

At the heart of Google's ranking system is a proprietary method called PageRank, and Google doesn't give detailed information about it. But Google does say this:

Google's order of results is automatically determined by more than 100 factors, including our PageRank algorithm.

Though we might never know exactly why results are different between the two search engines, at least we can have some fun spotting the differences— and end up with more search results than either one of the sites would have offered on its own.

One way to compare results is to simply open each site in a separate browser window and manually scan for differences. If you search for your favorite dog breed—say, australian shepherd—you'll find that the top few sites are the same across both Yahoo! and Google, but the two search engines quickly diverge into different results. At the time of this writing, both sites estimate exactly 1,030,000 total results for this particular query, but estimated result counts are sometimes a way to spot differences between the sites.

Viewing both sets of results in different windows is a bit tedious, and a clever Norwegian developer named Asgeir S. Nilsen has made the task easier at a site called Twingine.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Find Content You Can Reuse Legally

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Use the Yahoo! Creative Commons Search to find text and images with special licenses.

Most of the text, images, and audio you find through a Yahoo! Search are copyrighted materials. In fact, even if the author of a page or image doesn't specify that her work is copyrighted, a copyright still exists; by default, all works are copyrighted. This means you can never be sure how a particular author will react if you use part or all of her work in your own project.

Say you're preparing a public presentation and you'd like to use a photo of the Statue of Liberty. Unless you specifically pay for an image, you can't be sure the photographer of a photo you find on the Web won't sue you for copyright violation. Without contacting the photographer (or owner of the copyright), there's no way to know what he'll consider a fair use of his materials. While there is a legal concept called fair use that protects some use of copyrighted materials—especially for educational use—it's hard to know exactly which uses fall under this legal definition, because the concept of fair use is only vaguely defined.

This legal ambiguity is one of the reasons the nonprofit group Creative Commons (CC) has made several alternative licenses available to artists who want to license their work in a more specific way than a general copyright provides. For example, an artist can make her work available to anyone who wants to use his photo or text on the condition that it's used for noncommercial purposes, but the artist can still require payment for any commercial use. This means you could use a photo licensed in this way for a school report, but you'd need to pay the photographer if you were compiling a book you intended to sell. With a CC license, you know exactly how the artist would like her work to be used—or not used.

Another compelling aspect is that as long as you're following the conditions set forth in the license, you don't need to pay the artist to use the work—or even contact her asking to use the work. If for any reason you'd like to use the work in a way that's not covered by the license, however, you'll need to contact the artist. Basically, you just need to know how the Creative Commons licenses work to make sure you're playing by the rules.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Find Video from Across the Web

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The Yahoo! Video Search can help you find video clips that have been posted to the Web.

As Yahoo! Search scours the Web, it indexes not only the documents, text, and images it finds, but also video clips. Video files don't show up in a standard Yahoo! Web Search, but you can visit the Yahoo! Video Search to find clips from every corner of the Web.

To start finding videos, browse to http://video.yahoo.com (or type video search! into any Yahoo! Search form), enter a word or phrase, and click Search Video. Say you're interested in learning more about NASA's robotic vehicle for exploring Mars and you'd like to see the rover in action. You can find thumbnails of videos from across the Web by searching with the phrase mars rover, as shown in Figure 1-30.

Figure 1-30: Results of a Video Search for "mars rover"

Among the hundreds of results are clips from professional television news stories about the Mars rover, clips from NASA itself with video from the Mars rover, and amateur videos about the Mars rover. Yahoo! displays a single frame from the video, the name of the file, the video's dimensions, and the URL that contains a link to the video clip. To view a video, just click the thumbnail. The video will open from its original site within a Yahoo! frame, like the clip shown in Figure 1-31.

The top frame of a search result detail contains the video thumbnail, a link to play the video within your browser, and a link to send an email with a link to the clip. The bottom frame shows the page where the video was found, so you can see the video in its original context.

The videos you find through Yahoo! Video Search might be protected by copyright laws, so you'll need to be mindful of how you use any materials you find.

Figure 1-31: A Video Search result detail showing the video in context

As with a Yahoo! Web Search, there's an advanced video search form (http://video.search.yahoo.com/video/advanced). You can use the advanced search form to limit results to specific video formats, one of three sizes of videos— small, medium, or large—and videos that are longer or shorter than one minute. As with the Web Search, you can also limit your search to a specific site or adjust the SafeSearch features for the search.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Streamline Browsing with the Yahoo! Toolbar

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The Yahoo! Toolbar integrates Yahoo! with your web browser, letting you search, customize features, and access your mail from the browser at all times.

The Yahoo! Toolbar is a browser extension offered by Yahoo! that integrates many of Yahoo!'s features with your web browser. After installing the toolbar, you'll find an extra row of buttons in your browser, as shown in Figure 1-32.

Figure 1-32: The Yahoo! Toolbar

The toolbar contains buttons that perform Yahoo!-specific functions and a search form for quick access to Yahoo! searching. The toolbar can provide one-click access to all of your pages at Yahoo!, including your mail, groups, calendars, address book, and stock quotes. The toolbar is completely customizable, and you can choose which buttons and Yahoo! features are available on the toolbar. Because Yahoo! makes most of its features available via the toolbar, you can tailor your toolbar to suit the way you use Yahoo!.

To install the Yahoo! Toolbar, point your browser to http://toolbar.yahoo.com and click the orange Download button. From there, you'll find a page with instructions about downloading and installing the toolbar. At the time of this writing, the toolbar is available only for Internet Explorer, but there is a beta (i.e., testing) version available for Mozilla Firefox. Because the program is a browser extension rather than a traditional application, the download and installation will happen within the browser window. You'll need to approve some security requests along the way, and Yahoo! has laid out all of the steps to take on its site. Firefox requires you to restart the browser to see the toolbar, but Internet Explorer doesn't.

You can uninstall the Yahoo! Toolbar at any time by choosing Uninstall from the Toolbar Settings button.

Once the toolbar is installed, it will try to log in with your Yahoo! ID. If you are currently logged into Yahoo!, or if you have asked Yahoo! to "remember you," the toolbar should display the introductory Choose Buttons and Welcome Tour buttons. Otherwise, click the Sign In button and enter your Yahoo! ID and password.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Customize the Firefox Quick Search Box

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Though Yahoo! Web Search is a default option in the Firefox search box, with some quick coding you can add many other Yahoo! Search types.

If you use the Firefox web browser (available at http://www.mozilla.org/products/firefox), you're probably already aware of the useful search box in the upper-right corner. From any page, at any time, you can simply type a query into the box and press Enter to bring the search page up in the browser. Though Google is the default search engine, you can click the arrow to choose another search engine, as shown in Figure 1-43.

Figure 1-43: Firefox search box options

Yahoo! Web Search is a default option, and once you choose Yahoo! from the drop-down list, it will stay your top choice until you change it again.

The nice thing about this list of potential search engines is that you can add any search engine of your choice. In fact, Firefox offers an Add Engines…option that takes you to a page with more search choices you can install with a few clicks. The New Search Engines section contains a page (http://mycroft.mozdev.org/quick/yahoo.html) full of over 30 different Yahoo!-related searches you can add to the Firefox search box. These are searches that others have found useful and decided to share with the larger Mozilla community (Mozilla is the technology behind Firefox). The specialty Yahoo! searches include everything from searching Yahoo! Auctions and searching Yahoo! in different countries, to the Yahoo! Oxford Shakespeare reference. If you find yourself constantly looking for pithy quotes from The Tempest, adding this option to the Firefox search box could be the stuff dreams are made of.

To add a search engine from this Mozilla page, simply click the name of the search engine you'd like to add. A pop-up box will ask you to confirm your choice; click OK, and the new choice will be available in the Firefox search box menu. Behind the scenes, Firefox has copied a small .src file and icon to the searchplugins directory of the Firefox installation. This text file defines how the search works.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Spot Trends with Yahoo! Buzz

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Find out which pop culture phenomenon is on the way in or out by looking at trends in search queries.

Everywhere we look in the media, there are lists of bestsellers that can tell us what's hot at the moment. People watch the top-grossing movies like they'd watch the stock market, and best-selling books are always a topic of conversation. Yahoo! has its own way of tracking what's hot: by looking at the most common phrases people search for. If Yahoo! users are suddenly searching for singer Gwen Stefani more often than for Britney Spears, that shows a shift in interest or popularity.

Yahoo! tracks exactly these kinds of trends at the Yahoo! Buzz Index (http://buzz.yahoo.com). When you browse the Buzz Index, you'll find several Top Movers Charts, separated into categories such as TV, Music, Sports, Movies, Actors, Video Games, and overall queries. Each chart has the top 15 search queries with the greatest percentage increase for that particular day. These charts are a quick snapshot of which queries are gaining the most ground.

At the bottom of each chart, you'll find a "View complete chart…" link, which you can click to compare the top Movers with the Leaders. Instead of tracking the greatest change in terms of percentage, the Leaders chart shows you the top search queries for that category. Figure 1-45 shows the Leaders and Movers lists, side by side. The current champs (Leaders) are on the left, and the challengers (Movers) are on the right.

Figure 1-45: The overall Leaders and Movers at Yahoo! Buzz Index

Each entry on the Leaders chart shows the rank, whether the term is gaining or losing ground, how many points the term has moved in the last day, the number of days the term has been on the chart, and the overall buzz score. According to Yahoo!, a single buzz point is what 0.001% of Yahoo! users were searching for on a particular day. So the buzz score of 370 for the top entry in Figure 1-45 means that 0.37% of all Yahoo! users searched for American Idol on that day. With millions of users, that small percentage means thousands of people.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Find Hot Technologies at the Buzz Game

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The Buzz Game lets tech geeks speculate about where their industries are headed, and you can use their insider knowledge to spot tech trends.

If you follow technology trends like some people follow the stock market, you can put your knowledge to work by placing your bets on where technology is headed at the Yahoo! Buzz Game (http://buzz.research.yahoo.com). But even if you can't tell an iPod from a Typepad, the Yahoo! Buzz Game can help you make sense of the technology landscape.

A joint venture between Yahoo! Research and O'Reilly Media, Inc., the Yahoo! Buzz Game is a fantasy stock market for technology terms. The market attempts to predict what's going to be hot, by letting tech enthusiasts "buy" and "sell" technology terms. If the value of a particular tech term goes up, the score of players holding that term goes up. The players with the best scores can win prizes.

The Buzz Game is split into around 50 different markets, ranging from specific types of hardware and software, such as Portable Media Devices and Operating Systems, to less tangible ideas, such as Annoyances (which lets people bet on pop-up advertising or email spam).

Beyond letting you place bets and play the game, the Yahoo! Buzz Game site gives you a great snapshot of the current state of technology in a particular industry. For example, click Markets at the top of the home page and browse the categories for Portable Media Devices. At a glance, you can see the top contenders in the portable media gadget space. Figure 1-46 shows the relative market prices for various MP3 players in April 2005, with the iPod clearly ahead of the pack.

Clicking through each entry in a particular market, you can get a sense of how game players feel about each product's chances for success. If you browse through the items listed under Rumor Mill, you'll see products that don't exist yet and what tech-heads are predicting about their chances at existence.

Each company or product has a detail page, similar to a stock-tracking page, with the current price, its history, and overall

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Tame Long Yahoo! URLs

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

With an eye for URLs and the right tools, you can shorten long Yahoo! URLs when you need to send them via email.

Most of the time, we're all surfing the Web in virtual isolation. It's just you and the computer, and the last thing on your mind is the length of a URL at a page you're visiting. But as soon as you want to share the piece of the Web you're viewing with someone else, the length of a URL becomes important.

Because email programs wrap text at 72 characters (for easy reading), any URL that's longer could be broken. A broken URL means someone on the other end of the message won't be able to see the page you've sent them—or that they will have to spend a minute or two pasting the URL together in Notepad. And imagine trying to handwrite a note to someone that includes some of the URLs you stumble across!

Yahoo! has a lot of great content to share with others, but some of the URLs are definitely too long to send via email. Here's a Yahoo! Local URL for a page that shows a list of coffee shops in Sebastopol, CA:

	http://local.yahoo.com/results;_ylt=AvyPaC0wOiCme6J1PYb56tSHNcIF;_
	ylu=X3oDMTBtbGZ2dXFpBF9zAzk2NjEzNzY3BHNlYwNzZWFyY2g-?stx=coffee&
	csz=Sebastopol%2C+CA&fr=

Those 154 characters in the URL are definitely past the 72-character safe zone. If you take a look at the URL, you can see some variable/value pairs that contain the relevant information. The string ?stx=coffee looks important, as does csz=Sebastopol%2C+CA. But the rest of the URL looks like gibberish.

It's important to note that what looks like gibberish is actually useful information to Yahoo!, but it's not useful to you when you're trying to share links, so you can cut it out.

Cutting the garbage characters out of the URL will give you something more manageable:

	http://local.yahoo.com/results?stx=coffee&csz=Sebastopol%2C+CA

If the area you want to search is small enough, you can even use a Zip Code instead of the city and state combination:

	http://local.yahoo.com/results?stx=coffee&csz=95472

The 51 characters in this URL are well within the safe zone, and the URL points to exactly the same page. If you frequently find yourself sharing pages from Yahoo! Local, you might want to pick up a bookmarklet by Brian Cantoni (available at

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Opt Out of Advertiser Cookies

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Advertisers on Yahoo! might set browser cookies to track patterns, but you can tell them you'd rather not be tracked.

Browser cookies are an important piece of web technology that sites like Yahoo! rely on to give you a personalized experience at their site. Without cookies, Yahoo! wouldn't be able to let you choose your own news sources at My Yahoo!, watch your favorite stocks at Yahoo! Finance, or even send email with Yahoo! Mail. The cookie itself is simply a small bit of text that resides on your computer. When the cookie is set by a site such as Yahoo!, only that site can access the cookie's text in the future. The cookie lets Yahoo! know that a particular user is browsing its site, and Yahoo! can bring up saved preferences and settings for that particular user for future visits.

Cookies aren't inherently good or bad; such a judgment just depends on how they're used. You should be aware of how cookies are used, so that you can decide for yourself whether you want them to be set on your computer. You can view Yahoo!'s privacy policy regarding its use of cookies at http://privacy.yahoo.com.

In addition to the cookies that Yahoo! sets in order to remember your settings, Yahoo! allows other companies to set cookies through its web site. One of the ways Yahoo! makes money is through advertising, and Yahoo! sells sections of its pages to other companies. These other companies (called third parties) might set cookies that are in no way related to your Yahoo! account. In addition, these cookies do not have to follow the guidelines in Yahoo!'s privacy policy. This means that any advertiser or advertising network on Yahoo! might have its own privacy policy you should be aware of. Yahoo! lists more information about this practice on their Third Party and Affiliate Cookies on Yahoo! page (http://privacy.yahoo.com/privacy/us/adservers/details.html).

Most of these third-party cookies are probably being used to track patterns in Yahoo! usage. If a company has an advertisement on Yahoo! Finance, it might also want to know if those users are visiting Yahoo! Mail. These broad usage patterns help companies refine and target their campaigns without tracking any one person specifically. However, if you aren't thrilled with the idea of being tracked—even in a general way—across the sites you use, most of these advertising companies give you a way out.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Track News About Yahoo!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Yahoo! is a big company, and there are many ways to watch Yahoo! in action.

Whether you're a potential employee, investor, competitor, or just a fan of the site, you'll want to keep tabs on what Yahoo! is doing and where it may be headed. The news sources in this hack should give you a starting point for watching the company, and you can add the RSS feeds for the sources directly to My Yahoo! or your favorite newsreader. Once you subscribe to a few Yahoo!-related feeds, you won't have any trouble keeping up with the latest news.

You can use Yahoo! to find information about anything in the world, even Yahoo! itself:

Yahoo! News Search

Search over 7,000 news sources to find mentions of Yahoo! in news stories:

Web: http://search.news.yahoo.com/search/news/?p=Yahoo%21
RSS: http://news.search.yahoo.com/news/rss?ei=UTF-8&p=Yahoo%21

Yahoo! Finance Search

Track the progress of Yahoo!'s stock and find financial news and analysis at Yahoo! Finance:

Stock: http://finance.yahoo.com/q?s=YHOO
News: http://finance.yahoo.com/q/h?s=YHOO
News RSS: http://finance.yahoo.com/rss/headline?s=YHOO

Yahoo! Search Blog

This official weblog from Yahoo!, written by Yahoo! employees and special guest bloggers, gives you the inside scoop on what's new at Yahoo!:

Web: http://www.ysearchblog.com
RSS: http://www.ysearchblog.com/index.xml

Yahoo! Web Services Blog

Find out what's happening with Yahoo! Search Web Services at this official weblog:

Web: http://developer.yahoo.net/blog
RSS: http://developer.yahoo.net/blog/index.xml

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Spider the Yahoo! Catalog

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Writing a spider to spider an existing spider's site may seem convoluted, but it can prove useful when you're looking for location-based services. This hack walks through creating a framework for full-site spidering, including additional filters to lessen your load.

In this hack, you'll learn how to write a spider that crawls the Yahoo! group of portals. The choice of Yahoo! was obvious; because it is one of the largest Internet portals in existence, it can serve as an ideal example of how one goes about writing a portal spider.

But before we get to the gory details of code, let's define what exactly a portal spider is. While many may argue with such a classification, I maintain that a portal spider is a script that automatically downloads all documents from a preselected range of URLs found on the portal's site or a group of sites, as is the case with Yahoo!. A portal spider's main job is to walk from one document to another, extract URLs from downloaded HTML, process said URLs, and go to another document, repeating the cycle until it runs out of URLs to visit. Once you create code that describes such basic behavior, you can add additional functionality, turning your general portal spider into a specialized one.

Although writing a script that walks from one Yahoo! page to another sounds simple, it isn't, because there is no general pattern followed by all Yahoo! sites or sections within those sites. Furthermore, Yahoo! is not a single site with a nice link layout that can be described using a simple algorithm and a classic data structure. Instead, it is a collection of well over 30 thematic sites, each with its own document layout, naming conventions, and peculiarities in page design and URL patterns. For example, if you check links to the same directory section on different Yahoo! sites, you will find that some of them begin with http://www.yahoo.com/r, some begin with http://uk.yahoo.com/r/hp/dr, and others begin with http://kr.yahoo.com.

If you try to look for patterns, you will soon find yourself writing long if/ elsif/else sections that are hard to maintain and need to be rewritten every time Yahoo! makes a small change to one of its sites. If you follow that route, you will soon discover that you need to write hundreds of lines of code to describe every kind of behavior you want to build into your spider.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Browse the Yahoo! Directory

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

When you don't know exactly what you're looking for, the Yahoo! Directory might be able to help you find it.

Yahoo! started in 1994 as Jerry Yang and David Filo's organized list of favorite sites they'd found on the Web. Yahoo! has grown into much, much more, and many people think of Yahoo! as strictly a search company. Searching is great when you have a fairly good idea of what you're looking for, but the Yahoo! Directory is a great place when you'd rather browse.

There are two different kinds of shoppers, and they illustrate the difference between searching and browsing. Some shoppers know exactly what they're after and they want to find a store that carries that item, locate it in the store, and purchase it as quickly as possible. As with a web search, it helps to know a bit about what you're looking for if this is your style. Other shoppers want to explore a particular store, see what the store offers, and choose an item if the right one comes along. This style of browsing is suited to people who want to get a larger survey of items in a particular category before they necessarily decide what they're looking for.

Search forms are obviously built for searching. Directories are built for browsing. Unlike Yahoo! Search results, the Yahoo! Directory doesn't try to include every page it can find from across the Web. Instead, the sites listed in the directory are hand picked and reviewed by paid Yahoo! editors.

If you were interested in looking at a sampling of weblogs about politics, you might try a search at http://search.yahoo.com with the query political weblog. You would find political weblogs in the search results, along with news articles about political weblogs, college papers about political weblogs, and even pages that just mention the terms political and weblogs. But browsing the Political Weblogs category in the Yahoo! Directory (http://dir.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/Weblogs/Politics) will give you hundreds of links that have been selected by Yahoo! employees as being political weblogs.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Track Additions to Yahoo!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Keep track of the number of sites added to your favorite Yahoo! categories.

Every day, a squad of surfers at Yahoo! adds new sites to the Yahoo! index. These changes are reflected in the Yahoo! What's New page (http://dir.yahoo.com/new), along with the Picks of the Day.

If you're a casual surfer, you might not care about the number of new sites added to Yahoo!. But there are several scenarios when you might have an interest:

You regularly glean information about new sites from Yahoo!: Knowing which categories are growing and which categories are stagnant will tell you where to direct your attention.
You want to submit sites to Yahoo!: Are you going to spend your hard-earned money adding a site to a category where new sites are added constantly (meaning your submitted site might quickly get buried)? Or will you be paying to add to a category that sees few additions (meaning your site might have a better chance of standing out)?
You're interested in trend tracking: Which categories are consistently busy? Which are all but dead? By watching how Yahoo! adds sites to categories, over time you'll get a sense of the rhythms and trends and detect when unusual activity occurs in a category.

This hack scrapes the recent counts of additions to Yahoo! categories and prints them out, providing an at-a-glance look at additions to various categories. You'll also get a tab-delimited table of how many sites have been added to each category for each day. A tab-delimited file is excellent for importing into a spreadsheet, where you can turn the count numbers into a chart.

Save the following code to a file called hoocount.pl:

	#!/usr/bin/perl -w

	use strict;
	use Date::Manip;
	use LWP::Simple;
	use Getopt::Long;

	$ENV{TZ} = "GMT" if $^O eq "MSWin32";

	# the homepage for Yahoo!'s "What's New".
	my $new_url = "http://dir.yahoo.com/new/";
	
	# the major categories at Yahoo!. hash'd because
	# we'll use them to hold our counts string.
	my @categories = ("Arts & Humanities",		"Business & Economy",
				  "Computers & Internet",	"Education",
				  "Entertainment",				"Government",
				  "Health",				"News & Media",
				  "Recreation & Sports",	"Reference",
				  "Regional",					"Science",
				  "Social Science",				"Society & Culture");
	my %final_counts; # where we save our final readouts.

	# load in our options from the command line.
	my %opts; GetOptions(\%opts, "c|count=i");
	die unless $opts{c}; # count sites from past $i days.

	# if we've been told to count the number of new sites,
	# then we'll go through each of our main categories
	# for the last $i days and collate a result.

	# begin the header
	# for our import file.
	my $header = "Category";

	# from today, going backwards, get $i days.
	for (my $i=1; $i <= $opts{c}; $i++) {
	
		# create a Data::Manip time that will
		# be used to construct the last $i days	
		my $day; # query for Yahoo! retrieval.
		if ($i == 1) { $day = "yesterday"; }	
		else { $day = "$i days ago"; }
		my $date = UnixDate($day, "%Y%m%d");

		# and this date to
		# our import file.
		$header .= "\t$date";
		
		# and download the day.
		my $url = "$new_url$date.html";
		my $data = get($url) or die $!;
		
		# and loop through each of our categories.
		my $day_count; foreach my $category (sort @categories) {
			$data =~ /$category.*?(\d+)/; my $count = $1 || 0;
			$final_counts{$category} .= "\t$count"; # building our string.
		}
	}

	# with all our counts finished,
	# print out our final file.
	print $header . "\n";
	foreach my $category (@categories) {
		print $category, $final_counts{$category}, "\n";
	}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Yahoo! Directory Mindshare in Google

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

How does link popularity compare in Yahoo!'s searchable subject index versus Google's full-text index? Find out by calculating mindshare!

Yahoo! and Google are two very different animals. Yahoo! indexes only a site's main URL, title, and description, while Google builds full-text indexes of entire sites. Surely there's some interesting cross-pollination when you combine results from the two.

This hack scrapes all the URLs in a specified subcategory