Skip to Main Content
The Art of SEO
book

The Art of SEO

by Eric Enge, Stephan Spencer, Rand Fishkin, Jessie Stricchiola
October 2009
Beginner content levelBeginner
602 pages
19h 50m
English
O'Reilly Media, Inc.
Content preview from The Art of SEO

Benchmarking Current Indexing Status

The search engines have an enormous task—that of indexing the world’s online content; well, more or less. The reality is that they try hard to discover all of it, but they do not choose to include all of it in their index. This can happen for a variety of reasons, such as the page being inaccessible to the spider, being penalized, or not having enough link juice to merit inclusion.

When you launch a new site, or add new sections to an existing site, or are dealing with a very large site, not every page will necessarily make it into the index. To get a handle on this you will want to actively track the indexing level of your site. If your site is not fully indexed, it could be a sign of a problem (not enough links, poor site structure, etc.).

Getting basic indexation data from search engines is pretty easy. All three major search engines support the same basic syntax for that, which is site:yourdomain.com. Figure 4-12 shows a sample of the output from Bing.

Indexing data from Bing

Figure 4-12. Indexing data from Bing

Keeping a log of the level of indexation over time can help you understand how things are progressing. This can take the form of a simple spreadsheet.

Related to indexation is the crawl rate of the site. Google provides this data in Google Webmaster Central. Figure 4-13 shows a screen shot representative of the crawl rate charts that are available (another chart, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

The Art of SEO, 2nd Edition

The Art of SEO, 2nd Edition

Rand Fishkin, Eric Enge, Stephan Spencer, Jessie Stricchiola
The Art of SEO, 3rd Edition

The Art of SEO, 3rd Edition

Eric Enge, Stephan Spencer, Jessie Stricchiola
SEO Management

SEO Management

Véronique Duong

Publisher Resources

ISBN: 9780596809133Errata Page