There seems to be a big confusion between Sitemaps and a HTML site map for those new to websites. We are going to run through what each is and what they are used for today to try and undo the confusion.
First of all, notice the spelling of each. Sitemaps vs site map.
Sitemaps are XML files. They are meant for search bots and computer programs that understand the Sitemaps protocol (standard). Not all bots that roam the net understand the Sitemaps protocol. The 3 big players in the search industry do, Google, Bing and Yahoo! along with some others.
A site map on the other hand is a regular web page that humans, assistive technology and all search bots can read. It uses regular (X)HTML coding to present the information.
What are Sitemaps?
Here is the description from sitemaps.org (the site about Sitemaps):
In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.
The description goes on to say:
Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow crawlers that support Sitemaps to pick up all URLs in the Sitemap and learn about those URLs using the associated metadata. Using the Sitemap protocol does not guarantee that web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.
Notice where the text has been bolded?
Web crawlers usually discover pages from links within the site and from other sites.
That means your pages are found by the web crawlers (also known as search bots and spiders) through your internal linking and by finding links to your pages on other sites.
The first part is simple, build your
internal linking structure so
all search bots can find pages on the site other than the page they entered on. There is no guarantee a search bot or a human is going to arrive on the home page of your site. They can arrive on any page of the site through a search or from an incoming link somewhere.
The second part, incoming links, you can help along by having great content that ranks in the search results and then the word spreads that your site has fantastic content through
organic links. There are other ways to get incoming links but that is for another post.
Using the Sitemap protocol does not guarantee that web pages are included in search engines..
When a person with a new site asks how to get their site indexed an bunch of people will say to create a Sitemap and submit it to Google, Bing and Yahoo!. That is a misconception. The bolded text above clearly states that submtting a Sitemap does not guarantee the site or it’s pages will be included by the search engines which support the Sitemaps protocol (not all do).
Google themselves clearly state this also:
Creating and submitting a Sitemap helps make sure that Google knows about all the pages on your site, including URLs that may not be discoverable by Google’s normal crawling process….
…Google doesn’t guarantee that we’ll crawl or index all of your URLs.
When Should I Use Sitemaps?A small site doesn’t really need to use Sitemaps unless you are selling something and want to get your merchandise listed in a merchant/product search site that supports it as the file to feed your product information to their site.
Here is what Google says about when using Sitemaps is useful:
Sitemaps are particularly helpful if:
- Your site has dynamic content.
- Your site has pages that aren’t easily discovered by Googlebot during the crawl process—for example, pages featuring rich AJAX or images.
- Your site is new and has few links to it. (Googlebot crawls the web by following links from one page to another, so if your site isn’t well linked, it may be hard for us to discover it.)
- Your site has a large archive of content pages that are not well linked to each other, or are not linked at all.
About Sitemaps – Google Webmaster Tools Help
Ya, I know, Google listed if you have a new site it might help but remember, using Sitemaps does not guarantee your pages will be indexed. So lets look at site maps and how they work.
What is an (X)HTML Site Map?
An (X)HTML site map is a regular web page that can be read by any bot roaming the net, by those using assitive technology to surf the net and by regular web surfers.
The (X)HTML site map contains links to pages on your site. If you have a large site, you would list the main pages (e.g. category pages) of the site which takes you to a sub site map with links to pages within that category. For a small site, just one site map will be fine.
For search engine optimization and usability purposes, include a short description of what the person will see when they click the link. This will also help prevent your site map looking like a link farm page (page of just links).
Again for search engine optimization and usability purposes, a link to your site map should be included on each and every page. Remember, a bot or human can arrive on any page of your site so you want to make it easy for them to find their way around the site to discover other pages. Also remember, web crawlers find page through links so a site map is like a directory to the site with links for the web crawlers to follow.
When Should I Use a Site Map?
On every site! As already explained, a site map is accessible to everyone, human and bots. There are all kinds of bots roaming the net. Ones for directories, ones looking for reference sites to add to their site, all kinds! You can earn some free organic links just by providing a site map.
Ok not every site needs a site map to start with. If you are starting out small with 2, 3 or 4 pages a site map might be overkill at the beginning but plan for where the link will go in the future. As your site expands you can distroy the usability of your design by adding too many links to the navigation. Also too many links on a page compared to actual content can turn your pages into what looks like a link farm/spammy page.
For people who have accessibility challenges and those who just plain got lost on your site, a site map is a savour. They can just go to the site map to find where they want to go next on your site.
Which Should I do First? A Sitemap or a Site Map?
The site map! It’s the easiest to do and provides the most initial benefits: accessiblity, search engine optimization and usability.
But everyone says to do a Sitemap!
Well, even Matt Cutts, Google Spam Department Engineer says to do your site map first:
Sitemaps vs Site Map
When deciding which is better for your site, using Sitemaps or a site map, think beyond the search bots. You should always build your site for people first so that would include creating a site map first.
Later on, if your site gets really big or you decide to add your merchandise to a shopping search site, then tackle creating Sitemaps.
Submitting a Sitemap to the search engines is no guarantee that your pages will be indexed any faster than if you did not have one.
All it takes to attract the search bots is an incoming link from somewhere that gets visited by the search bots regularly which links to your site. The bot will find your site map and explore the rest of your site way quicker than you sitting around fretting and waiting for Google to review and index your Sitemap.
