An Introduction to Search Engines and Web Navigation

Chapter 2. THE WEB AND THE PROBLEM OF SEARCH

"Basically, our goal is to organise the world's information and make it universally accessible and useful."

—Larry Page, cofounder of Google

To understand the magnitude of the search problem we present some statistics regarding the size of the Web, its structure, and usage, and describe the important user activity of information seeking. We also discuss the specific challenges web search poses and compare local site search within an individual web site to global search over the entire web.

CHAPTER OBJECTIVES

• Give an indication of the size of the Web, and how it can be measured.

• Give an indication of the relative usage of search engines.

• Highlight the differences between structured data organized in tables, and traditional web data that does not have a fixed structure.

• Explain the bow-tie structure of the Web.

• Introduce the notion of a small-world network (or graph) in the context of the Web.

• Discuss different kinds of information-seeking strategies on the Web: direct navigation, navigating within a directory and using a search engine.

• Discuss the problems inherent in web information seeking.

• Introduce a taxonomy of web searches.

• Present the differences between web search and traditional information retrieval.

• Introduce the notions of precision and recall used to evaluate the quality of an information retrieval system, and discuss these in the context of web search.

• Discuss the differences between search within a local web site and global web search. ...