XML Sitemap Change Montioring

Automatically generated sitemaps can get forgotten in any project

An XML sitemap is a big list of all of the pages on your website that you want search engines to find and show in their search results. It’s only designed for search engines (not real people like you and me) so that they can easily crawl all of your pages.

What does an XML sitemap do?

An XML sitemap simply tells search engines which pages exist on your website so that they can go and crawl them. It’s basically a big list of URLs and when a search engine comes along, they can use the sitemap as a starting point to crawl those URLs. A human can look at an XML sitemap too, but you probably wouldn’t want to, it’s pretty boring. But that’s fine, pretty much all XML sitemaps are automatically generated and don’t need to look pretty, they just need to work and be as accurate as possible.

Why is an XML sitemap important?

Alongside having a really good navigation structure or internal linking, an XML sitemap is a great method to point search engine crawlers in the right direction. It can help give them a list of pages that you want them to crawl and index.

An XML sitemap should never replace good internal linking, it should just support it.

Having said that, an XML sitemap can also be really useful because you can also tell Google about it using Google Search Console. Once you’ve done this and they have processed the sitemap, you’ll be able to view some extra shiny reports which will help you understand if you have any indexing problems at all.

What can go wrong with an XML sitemap?

Sometimes, the pages that are listed in an XML sitemap may be broken or completely wrong. Ideally, you only want to include pages that you want Google to crawl and index, so if you accidentally include any that you don’t want them to see, you should make sure that you know this. By having pages included that you don’t care about, you may end up wasting crawl budget and Google may spend time crawling pages that you don’t really worry about and spend less time on the ones that you do.

Search engines can also be a bit grumpy and if they find things like 404 errors or 301 redirects in your sitemap, they may trust it a little less and not crawl it as often. So knowing when things change in your sitemap and going to investigate can help prevent these issues.