|
salam every one, this is a topic from google web master centrale blog: Webmaster level: All
Do you know how Google's crawler, Googlebot, handles conflicting directives in your robots.txt file? Do you know how to prevent a PDF file from being indexed? Do you know Googlebot’s favorite song? The answers to these questions (except for the last one :)), along with lots of other information about controlling the crawling and indexing of your site, are now available on code.google.com: Controlling crawling and indexing  Now site owners have a comprehensive resource where they can learn about robots.txt files, robots meta tags, and X-Robots-Tag HTTP header directives. Please share your comments, and if you have questions you can post them in our Webmaster Help Forum. this is a topic published in 2013... to get contents for your blog or your forum, just contact me at: devnasser@gmail.com |
Labels:
