XML Sitemaps

An XML sitemap is a file listing the pages on your site that you want search engines to know about. It is a direct way of saying "here is what exists", rather than relying entirely on crawlers finding everything by following links.

<loc> is the full URL. <lastmod> is when it last meaningfully changed — useful to engines deciding what to re-crawl, but only if it is honest. Setting every page's lastmod to today's date on every build teaches engines to ignore the field entirely.

• Tells engines about pages they might not otherwise discover — new pages, pages with few internal links
• Helps large sites get crawled more completely
• Signals which version of a URL you consider canonical
• Gives you a place to report against: Search Console will tell you how many submitted URLs were actually indexed

A hand-maintained sitemap goes out of date the first time someone forgets to update it, and a sitemap full of URLs that no longer exist is worse than none — it wastes crawl effort and signals carelessness.

Every serious content system can generate one automatically from its actual content. WordPress does it through plugins; modern frameworks generate it at build time or on request. Whatever the platform, the sitemap should be produced from the same source of truth as the site itself.

Two ways, and you should do both:

1. Reference it in robots.txt, which every crawler reads:
   Sitemap: https://example.com/sitemap.xml
2. Submit it directly in Google Search Console and Bing Webmaster Tools. This also unlocks the reporting on how many of its URLs got indexed.

Only URLs you actually want in search results, and only ones that return a normal 200 response. Leave out redirects, error pages, pages marked noindex, and non-canonical duplicates. A sitemap contradicting your other signals — listing a page you have also told engines not to index — is a confusing message, and engines resolve the contradiction however they choose.