|
salam every one, this is a topic from google web master centrale blog:
Confused about the best uses of robots.txt, nofollow, URL removal tool? Wondering how to keep some of your pages off the web? Our webspam lead, Matt Cutts, talks about the best ways to stop Google from crawling your content, and how to remove content from the Google index once we've crawled it. We love your feedback. Tell us what you think about this video in our Webmaster Help Group. * Note from Matt: Yes, robots.txt has been around since at least 1996, not 2006. It's hard for me to talk for 12-13 minutes without any miscues. :) |
Labels: crawling and indexing
