CHAPTER ONE

Search: Reflecting Consciousness and Connecting Commerce

Search has become integrated into the fabric of our society. With more than 197.9 billion searches performed worldwide each month as of April 2014, approximately 6.6 billion web searches are performed every day. This means that on average about 7,500 searches are performed every single second of every day.1 In addition, users have grown to expect that the responses to their search queries will be returned in less than one second.

If you have ever come across the various “SEO is dead” claims that make the rounds periodically in the digital marketing world, rest assured: SEO is both alive and well—and has not, as many feared, been “killed” by social media marketing as a driver of web traffic. In fact, in November 2014, Twitter announced that by renewing its focus on SEO, it increased the number of logged-out visitors coming to the site tenfold—from 7.5 million to 75 million visitors per month.2.

The high demand for search continues because people can now obtain in mere seconds information that 20 years ago would have required a trip to the library, the use of a card catalog and the Dewey Decimal System, and a physical search through halls of printed volumes—a process that could easily have consumed two hours or more. Through the new channel of search, people can conduct many of their shopping, banking, and social transactions online—something that has changed the way our global population lives and interacts.

This dramatic shift in behavior represents what investors like to label a disruptive event—an event that has changed something in a fundamental way. Search engines have been at the center of this disruptive event, and having a business’s website content appear prominently in the search engines when people are looking for the service, product, or resource that business provides is critical to its survival. As is the case with most paths to success, obtaining such prime search result real estate is not a simple matter, but it is one that this book aims to deconstruct and demystify as we examine, explain, and explore the ever-changing art of search engine optimization (SEO).

The Mission of Search Engines

Web searchers are free to use any of the many available search engines to find what they are seeking, so the burden is on the search engines to develop a relevant, fast, and fresh search experience. For the most part, search engines accomplish this by being perceived as having the most relevant results and delivering them the fastest, as users will go to the search engine they think will get them the answers they want in the least amount of time.

As a result, search engines invest a tremendous amount of time, energy, and capital in improving their relevance. This includes performing extensive studies of user responses to their search results, comparing their results against those of other search engines, conducting eye-tracking and click-through rate studies (discussed later in this chapter), and constructing PR and marketing campaigns.

Search engines generate revenue primarily through paid advertising. The great majority of this revenue comes from a pay-per-click (or cost-per-click) model, in which the advertisers pay only for users who click on their ads. Because the search engines’ success depends so greatly on the relevance of their search results, manipulations of search engine rankings that result in nonrelevant results (generally referred to as spam) are dealt with very seriously. Each major search engine employs a team of people who focus solely on finding and eliminating spam from their search results (generally referred to as “web spam” teams). This matters to SEO professionals because they need to be careful that the tactics they employ will not be considered spam by the search engines, as this would carry the risk of incurring penalties for the websites they work on.

The Market Share of Search Engines

Figure 1-1 shows the U.S. market share for search engines in June 2015, according to comScore. As you can see, Google is the dominant search engine on the Web in the United States.

Search engine market share (June 2014)
Figure 1-1. Search engine market share (June 2015)

In many European countries, the disparity is even greater. However, in some markets, Google is not dominant. In China, for instance, Baidu is the leading search engine. The fact remains, however, that in most world markets, a heavy focus on Google is a smart strategy for SEO.

The Human Goals of Searching

The basic goal of a human searcher is to obtain information relevant to a specific set of keywords and/or phrases entered into a search field, also known as a query. A searcher may formulate the query as a question, but the vast majority of searches are performed by users simply entering word combinations—leaving the search engines to do the work of determining “intent.” One of the most important elements to building an SEO strategy for a website is developing a thorough understanding of the psychology of your target audience, and how they use words and concepts to obtain information about the services and/or products you provide. Once you understand how the average searcher—and, more specifically, your target market—utilizes query-based search engines, you can more effectively reach and keep those users.

Search engine usage has evolved over the years, but the primary principles of conducting a search remain largely unchanged. Most search engine use consists of the following steps:

  1. Users experience the need for information. They may be looking for information on a specific website, and they will search for that website (a navigational query); they might want to buy something (a transactional query); or they might want to learn something (an informational query). We will discuss this in more detail in the following section.

  2. Users formulate that need using a string of words and phrases (comprising the query). Most people formulate their queries in one to three words, though as users are becoming more web savvy, they may use longer queries to generate more specific results more quickly. Table 1-1 gives a more detailed look at the percentages of searches per query length as of April 2014, the most recent comScore study on this aspect of search.

  3. Users execute the query, check the results, and if they seek additional information, will try a refined query.

Table 1-1. Searches by query length (comScore, April 2014 data)
Words Percent of searches
1 25.8%
2 22.8%
3 18.7%
4 13.2%
5+ 19.5%

When this process results in the satisfactory completion of a task, a positive experience is created for the user, the search engine, and the site providing the information or result.

Who Searches and What Do They Search For?

comScore reported that the number of search queries performed worldwide on the Web was approximately 197.9 billion across all search engines in April 2014.

comScore data also shows over 1.7 billion people were using a search engine on a given day in that month. Search engine users in the United States were slightly more likely to be women than men (50.3% versus 49.7%). According to comScore, about two-thirds of U.S. Internet users had an income of $40,000 or more (Table 1-2).

Table 1-2. Internet users by household income (May 2015)
U.S. household income Internet users
Less than $25,000 33,730 (13.2%)
$25,000–$39,999 32,941 (12.9%)
$40,000–$59,999 42,376 (16.5%)
$60,000–$74,999 27,861 (10.9%)
$75,000–$99,999 38,166 (14.9%)
$100,000 or more 81,015 (31.6%)

You can find additional data from studies, surveys, and white papers on Search Engine Land’s Search Behavior page.

All of this research data leads us to some important conclusions about web search and marketing through search engines. For example:

  • Search is a major source of valuable, targeted traffic.

  • Google is the dominant search player in most world markets.

  • Users tend to use shorter search phrases, but these are gradually getting longer, especially for non-navigational queries.

  • Search covers all types of markets.

Search is undoubtedly still one of the best and most important ways to reach consumers and build a business, regardless of that business’s size, reach, or focus.

Determining Searcher Intent: A Challenge for Search Marketers and Search Engines

Good marketers are empathetic, and smart SEO practitioners—as well as the search engines—share the goal of providing searchers with results that are relevant to their queries. Therefore, a crucial element to building an online marketing strategy around SEO and organic (sometimes referred to as “natural”) search visibility is understanding your audience and how they think about, discuss, and search for your service, product, and brand.

Search engine marketers need to be aware that search engines are tools—resources driven by intent toward a content destination. Using the search box is fundamentally different from entering a URL into the browser’s address bar, clicking on a bookmark, or clicking on a link to go to a website. Searches are performed with intent—the user wants to find specific information, rather than just land on it by happenstance; search is also different from browsing, or clicking around links on a web page. Because of this specificity, search traffic is generally of higher value to marketers than these other types of web traffic.

What follows is an examination of the different types of search queries and their categories, characteristics, and processes.

Informational Queries

Informational searches involve an incredibly broad range of queries. Consider the many types of information people might look for: local weather, driving directions, a celebrity’s recent interview, disease symptoms, self-help information, how to train for a specific type of career...the possibilities are as endless as the human capacity for thought. Informational searches are primarily non-transaction-oriented (although they can include researching information about a product or service); the information itself is the goal, and no interaction beyond clicking and reading is required for the searcher’s query to be satisfied. Figure 1-3 shows an example of an informational query.

Informational query
Figure 1-3. Informational query

Opportunities: Brand searchers with positive impressions of your site, information, company, and so on; attract inbound links; receive attention from journalists/researchers; potentially convert to sign up or purchase.

Average traffic value: The searcher may not be ready to buy anything just yet, or may not even have a long-term intent to buy anything, so the value tends to be “medium” at best. However, many of these searchers will later enter in a more targeted search, and this represents an opportunity to capture mindshare with those potential customers. For example, informational queries that are focused on researching commercial products or services can have high value.

Transactional Queries

Transactional searches don’t necessarily have to involve a credit card or immediate financial transaction. Creating a Pinterest account, signing up for a free trial account at DomainTools, or finding the best local Japanese restaurant for dinner tonight are all transactional queries. Figure 1-4 shows an example of a transactional query.

Transactional query
Figure 1-4. Transactional query

Opportunities: Achieve transaction (financial or other).

Average traffic value: Very high.

Research from Pennsylvania State University and the Queensland University of Technology shows that more than 80% of searches are informational in nature, and only about 10% of searches are navigational or transactional.3

The researchers went further and developed an algorithm to automatically classify searches by query type. When they tested the algorithm, they found that it was able to correctly classify queries 74% of the time. The difficulty in classifying the remaining queries was vague user intent—that is, the queries could have multiple meanings.

Searcher Intent

When you are building keyword research charts for clients or on your own sites, it can be incredibly valuable to determine the intent of each of your primary keywords. Table 1-3 shows some examples.

Table 1-3. Sample search queries and intent
Term Queries Intent Monetary value
Beijing Airport 980 Nav Low
Hotels in Xi’an 2,644 Info Mid
7-Day China tour package 127 Trans High
Sichuan jellyfish recipe 53 Info Low

This type of analysis can help you determine where to place ads and where to concentrate content and links.

Hopefully, this data can help you to think carefully about how to serve different kinds of searchers based on their individual intents, and how to concentrate your efforts in the best possible areas.

Although informational queries are less likely to immediately convert into sales, this does not mean you should forgo pursuing rankings on these queries; getting your informative content in front of users seeking information can be incredibly valuable, and can turn users into potential customers. If you are able to build a relationship with users who find your site after an informational query, they may be more likely to come to you to make a related purchase at a later date, and they may also decide to share your information with others via their own website, or through social media engagement—an indirect but potentially more valuable result than converting the single user into a paying customer.

One problem in search is that when most searchers formulate their search queries, their input is limited to just a handful of words (generally one to four). Because most people don’t have a keen understanding of how search engines work, they often provide queries that are too general or that are presented in a way that does not provide the search engine (or the marketer) with what it needs to determine, with 100% accuracy 100% of the time, their specific intent. Additionally, search engine users may not have a specific intent for an individual search query beyond “curiosity” about a general trending topic, or subject matter. While this poses potential difficulty in delivering relevant results, it also poses great opportunity to capture the mind of someone who may not know what he is looking for, specifically, but who takes an interest in the subsequent variety of results the search engine (and search marketers) deliver in response.

These types of general queries are important to most businesses because they often get the brand and site on the searcher’s radar, and this initiates the process of building trust with the user. Over time, the user will move on to more specific searches that are more transactional or navigational in nature.

If, for instance, companies buying pay-per-click (PPC) search ads bought only the high-converting navigational and transactional terms and left the informational ones to competitors, they would lose market share to those competitors. Over the course of several days, a searcher may start with digital cameras, hone in on Olympus OMD, and then ultimately buy from the store that showed up in her search for digital cameras and pointed her in the direction of the Olympus OMD model.

Given the general nature of how query sessions start, though, determining intent is quite difficult, and it can result in searches being performed where the user does not find what he wants—even after multiple tries. A July 2013 study conducted by ForeSee for the American Customer Satisfaction Index (ACSI) found that 77% of Google users and 76% of Bing users were satisfied with their experiences.

While 77% satisfaction is an amazing accomplishment given the complexity of building a search engine, this study still showed that more than 22% of users were not satisfied with their search results; additionally, these numbers represent the lowest levels of customer satisfaction with search engines since 2003, and could reflect users’ dissatisfaction with the number of ads that are infiltrating search engine results pages (SERPs). Greg Sterling of Search Engine Land provides a broad analysis of these issues (for the full discussion, see “Google Gets Lowest Score Ever in Customer Satisfaction Survey”). It will be interesting to see how these satisfaction numbers evolve over time. The important takeaway here is that in all instances, it is still difficult to determine exactly a searcher’s intent; and when the searcher’s intent is broad, there is ample opportunity to leverage your content assets with SEO.

As an SEO practitioner, you should be aware that some of the visitors that you succeed in attracting to your site may have arrived for the wrong reasons (i.e., they were really looking for something else), and these visitors are not likely to help your business goals. Part of your task as an SEO is to maintain a high level of relevance in the content placed on the pages you manage, to help minimize this level of waste—while still attempting to maximize SERP “mindshare” and gain brand exposure.

How Search Engines Drive Ecommerce

People make use of search engines for a wide variety of purposes, with some of the most popular being to research, locate, and buy products. Ecommerce sales reported by the U.S. Census Bureau were a healthy $80.3 billion in the first quarter of 2015.4 Forrester Research forecasts that U.S. ecommerce retail sales will reach $370 billion by 2017, outpacing sales growth at brick-and-mortar stores, as shown in Figure 1-8.

Forrester Research online retail forecast to 2017
Figure 1-8. Forrester Research online retail forecast to 2017

It is important to note that search and offline behavior have a heavy degree of interaction, with search playing a growing role in driving offline sales. In April 2014, Google, in an effort to solve the online search/offline conversion attribution dilemma, announced the launch of a pilot program within its AdWords system called “In-Store Attribution Transaction Reporting,” teaming up with large data providers Axciom Corp. and DataLogix Holdings Inc. to combine cookie data with offline personal information databases. It will be interesting to see how this program performs, and to what extent it helps search marketers close the attribution gap between search and brick-and-mortar conversions.

The Mobile Shift

In March 2015, eMarketer published a study showing that mobile advertising spend continues to increase at the expense of desktop advertising, and projects that by 2019, mobile ad spending will rise to $65.87 billion, or 72.2% of total U.S. digital advertising spend.5 Figure 1-9 shows the steep upward trend projection of mobile ad spend through 2019, reflecting consumers’ increasing use of mobile devices and tablets for search—and search marketers are taking notice.

eMarketer projected mobile ad spending through 2019
Figure 1-9. eMarketer projected mobile ad spending through 2019

The History of Eye Tracking: How Users Scan Results Pages

Way back in 2006, research firm Enquiro (now called Mediative) conducted heat-map testing with search engine users that produced fascinating results related to what users see and focus on when engaged in search activity.6 Figure 1-10 depicts a heat map showing a test performed on Google; the graphic indicates that users spent the most amount of time focusing their eyes in the upper-left area, where shading is the darkest. This has historically been referred to in search marketing as the “Golden Triangle.”

Enquiro eye-tracking results, 2006
Figure 1-10. Enquiro eye-tracking results, 2006

This particular study perfectly illustrated how little attention has traditionally been paid to results lower on the page versus those higher up, and how users’ eyes are drawn to bold keywords, titles, and descriptions in the organic results versus the paid search listings, which receive comparatively little attention. It also showed that different physical positioning of on-screen search results resulted in different user eye-tracking patterns. When viewing a standard Google results page, users tended to create this “F-shaped” pattern with their eye movements, focusing first and longest on the upper-left corner of the screen, then moving down vertically through the first two or three results, across the page to the first paid page result, down another few vertical results, and then across again to the second paid result. (This study was done only on left-to-right language search results—results for Chinese, Hebrew, and other non-left-to-right-reading languages would be different.)

In May 2008, Google introduced Universal Search (discussed in more detail in Chapter 10), which was a move from simply showing the 10 most relevant web pages (referred to as “10 blue links”) to showing other types of media, such as videos, images, news results, and so on, as part of the results of a basic web search. The other search engines followed suit within a few months, and the industry now refers to this general concept as blended search.

Blended search created more of a chunking effect of user eye attention, with the focus “chunks” hovering over various rich media objects such as images or video. Understandably, our eyes are drawn to the image first, then to the text beside it to see whether it corresponds to the image or video thumbnail. Based on an updated eye-tracking study published by Enquiro in September 2007, Figure 1-11 shows what the eye-tracking pattern on a blended search page looked like.

Enquiro eye-tracking results, blended search, 2007
Figure 1-11. Enquiro eye-tracking results, blended search, 2007

A January 2011 study performed by User Centric7 showed similar results, as shown in Figure 1-12.

User Centric eye-tracking results, 2011
Figure 1-12. User Centric eye-tracking results, 2011

In 2010, Enquiro investigated the impact of Google Instant on search usage and attention, noting that for queries in its study:

  • Percent of query typed decreased in 25% of the tasks, with no change in the others.

  • Query length increased in 17% of the tasks, with no change in the others.

  • Time to click decreased in 33% of the tasks and increased in 8% of the tasks.

In more recent studies, we can see how the continuously changing search results are shifting how users focus on a results page.

In October 2012, Mediative conducted a new eye-tracking study analyzing the areas of attention on Google Maps results pages for locally targeted searches, and as with a Google web search results page, users’ eyes were most drawn to the topmost search result, as shown in Figure 1-13.

Mediative eye-tracking results, 2012
Figure 1-13. Mediative eye-tracking results, 2012

A more recent study by Mediative, issued in 2014, called “The Evolution of Google Search Results Pages and Their Effect on User Behavior” shows us a fascinating progression away from the “Golden Triangle” as the search results—as well as search platforms (mobile!)—evolve. Figure 1-14 is the heatmap from this study for an updated Google SERP showing how user attention is moving more vertically than horizontally. Some have theorized that this shift in attention is due to the increased adoption of mobile search by mobile device users, while others think that the ever-changing SERP makeup that pushes the “blue text link” results lower on the page (e.g., the increase in the number prominence and content of top-of-page paid search ads, as well as universal and local listings for various queries) is contributing to this change.

Google SERP eye-tracking results, 2014: users have moved from horizontal to vertical scanning
Figure 1-14. Google SERP eye-tracking results, 2014: users have moved from horizontal to vertical scanning

Another notable trend in user eye attention is that because the #1 organic site’s actual position on the SERP is moving lower, sites that are positioned lower on the results page are seeing more click activity than in years past—making ranking in organic sites at these lower positions more valuable than in the past as well. Figure 1-15 highlights the increased attention being placed lower on the page.

Google SERP, eye-tracking results, 2014: lower SERPs get more attention
Figure 1-15. Google SERP, eye-tracking results, 2014: lower SERPs get more attention

These types of studies are a vivid reminder of how important search engine results pages really are. And, as the eye-tracking research demonstrates, as search engines—and therefore search results—continue to evolve, users’ search and engagement patterns will follow suit. There will be more items on the page for searchers to focus on, more ways for searchers to remember and access the search listings, and more interactive, location-based delivery methods and results layouts—which will keep changing as other search environments and platforms, such as mobile, continue to evolve.

Figure 1-16 shows a “Mega SERP” published by Moz: a display of the current universe of potential results within a Google SERP.

Moz mega-SERP
Figure 1-16. Moz mega-SERP

Click Tracking: How Users Click on Results (Organic Versus Paid)

By now, you should be convinced that you want your site content to be prominently displayed within SERPs. It never hurts to be #1 in the organic search results.

In contrast, data shows that you may not want to be #1 in the paid search results, because the resulting cost to gain the #1 position in a PPC campaign can reduce the total net margin on your campaign. A study released by AdGooroo in June 20088 found that:

Bidding for top positions usually makes financial sense only for high-budget, brand-name advertisers. Most other advertisers will find the optimal position for the majority of their keywords to lie between positions 5–7.

Of course, many advertisers may seek the #1 position in paid search results, for a number of reasons. For example, if they have a really solid backend on their website and are able to make money when they are in the #1 position, they may well choose to pursue it. Nonetheless, the data from the survey suggests that there are many organizations for which being #1 in paid search does not make sense.

Even if your organic ranking is #1, you can still increase the ranking page’s click rate by having a sponsored ad above it or in the righthand column. The AdGooroo survey showed that having a prominent paid ad on the same search results page makes your #1 organic ranking receive as much as 20% more clicks.

Distribution of Search Results and Traffic

To start breaking this down a bit, Figure 1-17 shows the screen real estate occupied by the two types of search results. This screenshot was taken prior to Google’s January 2012 “Search, plus Your World” announcement, but it’s the type of screen layout related to studies that will help you understand which portions of the search results receive the most clicks.

This example from Google shows how the paid results appear above and to the right of the organic search results. Note that Google often does not show paid results above the organic results, in which case the paid results show up only on the right.

Your position in the results has a huge impact on the traffic you will receive. Studies on the impact of SERP position have shown widely varying results, but do agree that the advantage of higher positions is significant.

In 2013, Catalyst Search Marketing published the most recent comprehensive study of Google CTR (click-through rate), which shows CTR data for both desktop and mobile searches, as well as for branded and unbranded searches.9 In this study, Catalyst found that the top four Google SERP positions received 83% of first-page organic clicks, and that 48% of searches result in a first-page organic click. Table 1-5 shows the average CTR (percentage of clicks received) by Google SERP position.

Table 1-5. Google CTR by position
Average position Average CTR
1.00 17.16
2.00 9.94
3.00 7.64
4.00 5.31
5.00 3.50
6.00 1.63
7.00 1.09
8.00 1.04
9.00 0.44
10.00 0.51

This data provides insight around platform CTR differentiators (desktop versus mobile), searcher intent CTR differentiators (branded versus unbranded), and query length CTR differentiators (one, two, three, and four-plus-word query data).

In Table 1-6, you can see that the more refined a user’s search query is—that is, the more words used in the query—the greater the likelihood that the user will click on search results farther down the page.

Table 1-6. Google CTR by position and query length
Average position One word Two words Three words Four+ words
1.00 17.68 16.32 16.28 18.76
2.00 4.41 8.30 10.58 12.41
3.00 3.56 6.86 7.93 8.64
4.00 1.50 4.10 5.12 6.38
5.00 1.86 2.64 3.61 4.11
6.00 0.37 1.10 1.60 2.18
7.00 0.00 1.08 1.12 1.36
8.00 0.00 1.13 1.45 0.84
9.00 0.08 0.52 0.36 0.53
10.00 0.44 0.42 0.54 0.62

As the search engines, search engine results pages, and searcher behavior evolve, the practice of search engine optimization must evolve in tandem.

Conclusion

Search has penetrated the very fabric of global society. The way people learn, work, share, play, shop, research, socialize, and interact has changed forever, and organizations, causes, brands, charities, individuals—almost all entities—need to have an Internet presence of some kind, and they need the search engines and search functionality within all platforms to generate exposure and facilitate engagement. As our society moves ever closer to a professional consumer (“prosumer”) economy, the ways in which people create, publish, distribute, and ultimately find information and resources on the Web will continue to be of great importance. This book will investigate further just how search, and therefore search engine optimization, is at the center of the web ecosystem and thus is your key to success in the ever-evolving digital economy.

1 comScore, “comScore Releases April 2014 U.S. Search Engine Rankings,” May 16, 2014, http://www.comscore.com/Insights/Market-Rankings/comScore-Releases-April-2014-US-Search-Engine-Rankings.

2 Danny Sullivan, “Twitter: Renewed Focus On SEO Generated 10 Times More Visitors,” November 12, 2014, http://searchengineland.com/twitter-seo-more-visitors-208160.

3 Bernard J. Jansen, Danielle L. Booth, and Amanda Spink, “Determining the Informational, Navigational, and Transactional Intent of Web Queries,” Information Processing and Management 44 (2008) 1251–1266, http://bit.ly/jansen_user_intent.

4 U.S. Census Bureau News, “Quarterly Retail E-Commerce Sales, 1st Quarter 2015,” May 15, 2015, http://bit.ly/ecommerce_q1_2015.

5 eMarketer, “Mobile Will Account for 72% of US Digital Ad Spend by 2019: Consumer Usage and Better Ad Formats Drive Dollars to Mobile Apps,” March 24, 2015, http://bit.ly/mobile_ad_spend.

6 Mediative, published by Enquiro Search Solutions. “Eye Tracking Report 2: Google, MSN, and Yahoo! Compared!” November 2006, http://bit.ly/eye_tracking_paper.

7 Aga Bojko, “Eye Tracking Bing Vs. Google: A Second Look,” GfK Insights Blog, January 27, 2011, http://blog.gfk.com/2011/01/eye-tracking-bing-vs-google-a-second-look/.

8 AdGooroo, “Research Update: How Keyword Length and Ad Position Impact Clickthrough Rate and Cost-per-click on Google Adwords”. June 2008.

9 Catalyst, “Google CTR Study,” http://www.catalystsearchmarketing.com/pubs/google-ctr-study/.

Get The Art of SEO, 3rd Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.