|
Seo Master present to you:
 By default each website allows the Search engines robots however if you would like to restricts the robots either to not crawl any bound directory, file or the complete website then you may want the robots.txt file in which you've got to write instructions for search engine bots. Steps to edit robots.txt on bloggerYour Site Settings › Search preferences › Crawlers and indexing Here you will able to see two options Custom robots.txt and Custom robots header tags.These two options would offer you the flexibility to customise your robot.txt file.In the last post i told you about Custom robots header tags. Now press edit button which is present after 'Custom robots.txt' option. After pressing on edit button you can see a message " Enable custom robots.txt content? " so press "Yes" and proceed to next step.  Click on Save Changes button.  You are done! How to block Link from search engines? No, you have to write the URL. For example, if your want to stop robots from crawling this URL (www.www.matrixar.com/p/about.html) then, in the robot.txt file you will enter this command. User-agent: *Disallow: p/about.htmlYour robots.txt file located under your main blogspot directory as for Blogger Known its located on the following url: http://www.www.matrixar.com/robots.txtBy default blogger robots.txt contains the following content: User-agent: Mediapartners-GoogleI know this post is not a new thing but I hope it might help many newbies to understand the importance of robots.txt. |
Labels: Blogger Tips, Search Engines, seo, Tutorial
