MatrixAdapt | Logiciel de gestion d'Entreprise, Création et référencement des sites web

salam every one, this is a topic from google web master centrale blog: I've seen a lot of questions lately about robots.txt files and Googlebot's behavior. Last week at SES, I spoke on a new panel called the Bot Obedience course. And a few days ago, some other Googlers and I fielded questions on the WebmasterWorld forums. Here are some of the questions we got:

If my site is down for maintenance, how can I tell Googlebot to come back later rather than to index the "down for maintenance" page?
You should configure your server to return a status of 503 (network unavailable) rather than 200 (successful). That lets Googlebot know to try the pages again later.

What should I do if Googlebot is crawling my site too much?
You can contact us -- we'll work with you to make sure we don't overwhelm your server's bandwidth. We're experimenting with a feature in our webmaster tools for you to provide input on your crawl rate, and have gotten great feedback so far, so we hope to offer it to everyone soon.

Is it better to use the meta robots tag or a robots.txt file?
Googlebot obeys either, but meta tags apply to single pages only. If you have a number of pages you want to exclude from crawling, you can structure your site in such a way that you can easily use a robots.txt file to block those pages (for instance, put the pages into a single directory).

If my robots.txt file contains a directive for all bots as well as a specific directive for Googlebot, how does Googlebot interpret the line addressed to all bots?
If your robots.txt file contains a generic or weak directive plus a directive specifically for Googlebot, Googlebot obeys the lines specifically directed at it.

For instance, for this robots.txt file:

User-agent: *
Disallow: /

User-agent: Googlebot
Disallow: /cgi-bin/

Googlebot will crawl everything in the site other than pages in the cgi-bin directory.

For this robots.txt file:

User-agent: *
Disallow: /

Googlebot won't crawl any pages of the site.

If you're not sure how Googlebot will interpret your robots.txt file, you can use our robots.txt analysis tool to test it. You can also test how Googlebot will interpret changes to the file.

For complete information on how Googlebot and Google's other user agents treat robots.txt files, see our webmaster help center.this is a topic published in 2013... to get contents for your blog or your forum, just contact me at: devnasser@gmail.com

Seo Master present to you: By Louis Gray, Developer Relations Team

We think of Google I/O as a place where our technologies come to life. Over the course of our three-day conference next week, you will see new technologies demoed for the first time, learn from the Googlers behind the code and meet with Sandbox partners who’ve built businesses using our APIs. And while I/O Live makes Google I/O accessible to every single developer around the world, we want to bring you knowledge, tools, and people from the Google Developers community, all year round–not just three days in June.

We think connecting you with Google's experts and your fellow developers can inspire incredible app creation. So today, we're introducing Google Developers Live, a destination for developers around the world that will feature live, interactive broadcasts ranging from developer-focused game shows to Office Hours where you can connect with the engineers who created and work on your favorite Google product.

Google Developers Live allows us to bring you the excitement of Google I/O year-round, beginning today with the release of starter-level sessions on everything from Android to YouTube, to help prepare you for the more advanced content that will be presented next week. And, if you have any questions after watching these tutorials, we've set up Office Hours with each of our presenters on Google Developers Live so you can ask them directly.

Tune in to Google Developers Live at http://developers.google.com/live and start filling your calendar with Office Hours, App Reviews and more from Google's many products, along with original programming. Whether you're looking for Android, Chrome, Google+, Drive, Cloud Platform, Google Maps, YouTube or something else, we’ve got a hangout for you - all year round.


Louis Gray is a Program Manager on Google's Developer Relations Team, running Google Developers Live. He eats, sleeps and breathes Google products and APIs.

Posted by Scott Knaster, Editor

2013, By: Seo Master

salam every one, this is a topic from google web master centrale blog: Posted by Charlene Perez, Search Quality Teamthis is a topic published in 2013... to get contents for your blog or your forum, just contact me at: devnasser@gmail.com