In This Chapter
Getting to know the servers
Making sure your server is healthy and fast
Excluding pages or sites from the search engines
Passing instructions to search engines with a robots text file
Meta robots tags
Building a customized 404 error page
Avoiding dirty IPs and bad neighborhoods
Your Web server is the software application/service that runs your Web site. (The term Web server can be used to refer to both the hardware and the software that runs a Web site, but here we're talking about the software.) Anytime a user does something on your site, such as load a page or view an image, it's your Web server that receives the request and serves up what the user wants. Like a good waiter in a restaurant, your site's server should be as fast and efficient as possible so that your site visitors feel happy and well satisfied.
Server issues impact search engine ranking, from the type of server you use to how well it performs. Sites that frustrate users by being slow or unavailable are not the type of sites that any search engine wants to present in its results. A slow server, or a server that fails often, can cause a site to drop out of the search engine's index (the databases of Web site content that Google, Yahoo!, or Microsoft Live Search pulls from when delivering search results), or prevent a site from ever being indexed in the first place. A key and yet often overlooked point of failure for a Web site is the server environment ...