The Art of SEO, 2nd Edition

Determining Top Competitors

Understanding the competition should be a key component of planning your SEO strategy. The first step is to understand who your competitors in the search results really are. It can often be small players who give you a run for your money. For example, consider the previously mentioned credit card search in Google (Figure 4-1); Visa, MasterCard, American Express, and Discover Card all fail to reach the #1 position in the Google results.

Instead, affiliate players dominate these results. Affiliates tend to be the most adept at search engine optimization and can be the most lax in abiding by the search engines’ terms and conditions.

Two Spam Examples

Affiliates that cheat tend to come and go out of the top search results, as only sites that implement ethical tactics are likely to maintain their positions over time. You can help expedite the cheaters’ fall from grace by reporting them to Google at http://www.google.com/contact/spamreport.html, or better yet, via the dashboard in your Google Webmaster Tools account (where your report will carry more weight).

How do you know whether a top-ranking site is playing by the rules? Look for dubious links to the site using a backlink analysis tool such as Open Site Explorer. Since the number of links is one factor search engines use to determine search position, few ethical websites will attempt to obtain links from a multitude of irrelevant and low-quality sites.

This sort of sleuthing can reveal some surprises. For instance, here are examples of two devious link schemes:

• GiftCertificates.com’s short-lived nemesis was FindGiftCards.com, which came out of nowhere to command the top two spots in Google for the all-important search term gift certificates, thus relegating GiftCertificates.com to the third position. How did FindGiftCards.com do it? It operated a sister site, 123counters.com, with a free hit counter that propagated “link spam” across thousands of sites, all linking back to FindGiftCards.com and other sites in its network.

  Sadly for FindGiftCards.com, Stephan Spencer, founder and president of the e-marketing agency Netconcepts, outed the company in an article he wrote for Multichannel Merchant back in 2004 (http://multichannelmerchant.com/catalogage/ar/marketing_casing_competition/), and Google became aware of the scam. The end result? The site was knocked down to only two pages in the Google index, as shown in Figure 4-9.

• CraigPadoa.com was a thorn in the side of SharperImage.com, outranking the latter for its most popular product, the Ionic Breeze, by frameset trickery and guestbook spamming (in other words, defacing vulnerable websites with fake guestbook entries that contained spammy links back to its own site). As soon as The Sharper Image realized what was happening, it jumped on the wayward affiliate. It also restricted such practices in its affiliate agreement and stepped up its monitoring for these spam practices.

Figure 4-9. Site with only two pages in the index

Seeking the Best

Look for competitors whose efforts you would like to emulate (or “embrace and extend,” as Bill Gates would put it)—usually a website that consistently dominates the upper half of the first page of search results in the search engines for a range of important keywords that are popular and relevant to your target audience. Note that your “mentor” competitors shouldn’t just be good performers; they should also demonstrate that they know what they’re doing when it comes to SEO. To assess competitors’ competence at SEO, you need to answer the following questions:

• Are their websites fully indexed by Google and Yahoo!? In other words, are all their web pages, including product pages, making it into the search engines’ databases? You can go to each search engine and type in site:theirdomain.com to find out. A competitor with only a small percentage of its site indexed in Google probably has a site that is unfriendly to search spiders.

• Do their product and category pages have keyword-rich page titles (title tags) that are unique to each page? You can easily review an entire site’s page titles within Google or Yahoo! by searching for site:www.yourcompetitor.com.

  Incidentally, this type of search can sometimes yield confidential information. A lot of webmasters do not realize that Google has discovered and indexed commercially sensitive content buried deep in their sites. For example, a Google search for confidential business plan filetype:doc will yield a lot of real business plans among the sample templates.

• Do their product and category pages have reasonably high PageRank scores?

• Is anchor text across the site, particularly in the navigation, keyword-rich?

• Are the websites getting penalized? You can overdo SEO. Too much keyword repetition or too many suspiciously well-optimized text links can yield a penalty for over-optimization. Sites can also be penalized for extensive amounts of duplicate content. You can learn more about how to identify search engine penalties in the section “Content Theft” in Chapter 11.

• Are they spamming the search engines with “doorway pages”? According to Google: “Doorway pages are typically large sets of poor-quality pages where each page is optimized for a specific keyword or phrase. In many cases, doorway pages are written to rank for a particular phrase and then funnel users to a single destination” (http://www.google.com/support/webmasters/bin/answer.py?answer=66355).

Uncovering Their Secrets

Let’s assume your investigation has led you to identify several competitors who are gaining excellent search placement using legitimate, intelligent tactics. Now it is time to uncover their secrets:

• What keywords are they targeting? You can determine this by looking at the page titles (up in the bar above the address bar at the top of your web browser, which also appears in the search results listings) of each competitor’s home page and product category pages, then by looking at their meta keywords tag (right-click, select View Source, and then scour the HTML source for the list of keywords that follow the bit of HTML that looks something like the following:

  <meta name="keywords" content="keyword1, keyword2, ...">

• Who’s linking to their home page, or to their top-selling product pages and category pages? A link popularity checker can be quite helpful in analyzing this.

• If it is a database-driven site, what technology tricks are they using to get search engine spiders such as Googlebot to cope with the site being dynamic? Nearly all the technology tricks are tied to the ecommerce platforms the competitors are running. You can check to see whether they are using the same server software as you by using the “What’s that site running?” tool at the top-left corner of http://news.netcraft.com. Figure 4-10 shows a screenshot of the results for HSN.com.

  

  Figure 4-10. Sample Netcraft output

  While you are at it, look at “cached” (archived) versions of your competitors’ pages by clicking on the Cached link next to their search results in Google to see whether they’re doing anything too aggressive, such as cloaking, where they serve up a different version of the page to search engine spiders than to human visitors. The cached page will show you what the search engine actually saw, and you can see how it differs from the page you see when you go to the web page yourself.

• What effect will their future SEO initiatives have on their site traffic? Assess the success of their SEO not just by the lift in rankings. Periodically record key SEO metrics over time—the number of pages indexed, the PageRank score, the number of links—and watch the resulting effect on their site traffic.

  You do not need access to competitors’ analytics data or server logs to get an idea of how much traffic they are getting. Simply go to Compete.com, Quantcast.com, or Alexa.com and search on the competitor’s domain. If you have the budget for higher-end competitive intelligence tools, you can use comScore.com or Hitwise.com.

  The data these tools can provide is limited in its accuracy, but still very useful in giving you a general assessment of where your competitors are. The tools are most useful when making relative comparisons between sites in the same market space. To get an even better idea of where you stand, use their capabilities to compare the traffic of multiple sites. In this mode, you can get a pretty accurate idea as to how your traffic compares to theirs.

  You can now get this type of data directly from Google as well, using Google Trends for Websites. The output of this tool is just a summary of Google traffic, but it is a much larger data set than is available from the other products. Figure 4-11 shows an example of the output from Google Trends for Websites.

  

  Figure 4-11. Google Trends for Websites

  Note that tools such as Alexa, Compete, and Quantcast do have other unique features and functionality not available in Google Trends for Websites.

• How does the current state of their sites’ SEO compare with those of years past? You can reach back into history and access previous versions of your competitors’ home pages and view the HTML source to see which optimization tactics they were employing back then. The Wayback Machine (http://www.archive.org) provides an amazingly extensive archive of web pages.