Skip to Content
Google Hacks
book

Google Hacks

by Tara Calishain, Rael Dornfest
February 2003
Intermediate to advanced
352 pages
9h
English
O'Reilly Media, Inc.
Content preview from Google Hacks

Removing Your Materials from Google

How to remove your content from Google’s various web properties.

Some people are more than thrilled to have Google’s properties index their sites. Other folks don’t want the Google bot anywhere near them. If you fall into the latter category and the bot’s already done its worst, there are several things you can do to remove your materials from Google’s index. Each of Google’s properties—Web Search, Google Images, and Google Groups—has its own set of methodologies.

Google’s Web Search

Here are several tips to avoid being listed.

Making sure your pages never get there to begin with

While you can take steps to remove your content from the Google index after the fact, it’s always much easier to make sure the content is never found and indexed in the first place.

Google’s crawler obeys the “robot exclusion protocol,” a set of instructions you put on your web site that tells the crawler how to behave when it comes to your content. You can implement these instructions in two ways: via a META tag that you put on each page (handy when you want to restrict access to only certain pages or certain types of content) or via a robots.txt file that you insert in your root directory (handy when you want to block some spiders completely or want to restrict access to kinds or directories of content). You can get more information about the robots exclusion protocol and how to implement it at http://www.robotstxt.org/.

Removing your pages after they’re indexed

There are ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

Google Hacks, 2nd Edition

Google Hacks, 2nd Edition

Tara Calishain, Rael Dornfest
Google Hacks, 3rd Edition

Google Hacks, 3rd Edition

Rael Dornfest, Paul Bausch, Tara Calishain
Open source at AWS

Open source at AWS

Adrian Cockcroft
Algorithms of the Intelligent Web

Algorithms of the Intelligent Web

Dmitry Babenko, Haralambos Marmanis

Publisher Resources

ISBN: 0596004478