Tutorial · Practitioner
XML Sitemaps for AI Discovery
An XML sitemap lists the canonical URLs you want crawled and when each last changed. Google and Bing both ignore `priority` and `changefreq`, and both rely on an accurate `lastmod` ([Google](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap), [Bing](https://blogs.bing.com/webmaster/2025/7/Keeping-Content-Discoverable-with-Sitemaps-in-AI-Powered-Search/)). Bing says accurate lastmod helps updates appear faster in search results and AI-generated answers, which makes sitemaps a direct input into Copilot freshness.
Facts are from Google Search Central and Bing Webmaster documentation, accessed 30 Oct 2026. Points marked "Our view" are interpretation.
On this page
- What are the rules?
- How should lastmod be set?
- How do sitemaps relate to AI answers?
- How do you submit a sitemap?
- What does a good entry look like?
- How do you audit your sitemap?
- What goes wrong? (Troubleshooting)
- How do you automate it?
- Exercise
- Checklist
- From MentionX data: which pages need to be discoverable
- Worked example: Ledgerline fixes a stale sitemap
- Which URLs should you leave out?
- How large can a sitemap be?
- Do changefreq and priority matter?
- How do sitemaps help AI search?
- Should you use IndexNow?
- How do you check sitemap health?
- What about image and video sitemaps?
- How do you handle pages that are removed?
- Which tools generate sitemaps?
- Where should the sitemap live?
- How do multilingual sites handle sitemaps?
- What should a sitemap review cover after a migration?
- Who should own the sitemap?
- Common questions
- Related
- Sources
Checked on 31 October 2026. "Our view" marks MentionX opinion. Ledgerline is a fictional company used for illustration.
What are the rules?
| Rule | Bing | |
|---|---|---|
| Size limit | 50 MB uncompressed or 50,000 URLs per sitemap; split larger ones | Follows the sitemaps protocol |
| Encoding | UTF-8 | — |
priority, changefreq | Ignored | Ignored |
lastmod | Used if consistently and verifiably accurate | Key signal for recrawl and reindex |
| URLs to list | — | Only canonical URLs; remove deleted or redirected ones promptly |
Sources: Google, Bing, Bing guidelines.
How should lastmod be set?
Google says lastmod should reflect the date and time of the last significant update: changes to the main content, structured data or links count; a copyright-date update does not (Google).
- Use ISO 8601 with both date and time, e.g.
2026-10-30T09:15:00+01:00. - Don't set lastmod to the time the sitemap was generated unless the page actually changed.
- If dates are consistently set to the current date, Bing says it will suspect their validity and may disregard them.
- Bing's 2023 study found many sitemaps used the generation date rather than the content-modification date.
How do sitemaps relate to AI answers?
Bing says freshness signals directly influence how quickly updates are reflected in search results and AI-generated answers, and that lastmod helps Bing prioritise URLs for recrawling or skip unchanged ones (Bing). Bing's webmaster guidelines state that Bing and Copilot rely on the same crawling, indexing and ranking foundation as traditional search, and that proper sitemap usage supports accurate indexing and grounding eligibility (Bing).
Our view: Google doesn't make an equivalent AI-specific statement, but AI Overviews draw on Google's index, so sitemap hygiene matters there too. Sitemaps don't guarantee visibility; Bing says so explicitly.
How do you submit a sitemap?
- Reference it in robots.txt with a line such as
Sitemap: https://www.example.com/sitemap.xml. Google says you can list multiple sitemaps there, with no limit (Google). - Submit in Search Console and Bing Webmaster Tools to see processing status and errors.
- Use a sitemap index if you have more than 50,000 URLs.
- Pair with IndexNow for instant change notifications to participating engines; see IndexNow for Bing, Copilot and ChatGPT.
What does a good entry look like?
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://www.example.com/learn/technical/xml-sitemaps-ai-discovery</loc>
<lastmod>2026-10-30T09:15:00+01:00</lastmod>
</url>
</urlset>Only loc and an honest lastmod. No priority or changefreq needed.
How do you audit your sitemap?
- Fetch the sitemap and count URLs and files.
- Request every URL; flag anything not returning 200.
- Flag URLs whose canonical points elsewhere (canonicals guide).
- Flag noindexed URLs.
- Compare lastmod with the page's real update date for a sample.
- Check that important pages, such as new guides, are present.
Expected result: every URL returns 200, is self-canonical, indexable and has a believable lastmod.
What goes wrong? (Troubleshooting)
| Symptom | Likely cause | Fix |
|---|---|---|
| All lastmod values equal today | Generation date used | Use real content-change time |
| Redirected URLs in sitemap | Not cleaned after migrations | Regenerate from canonical URLs only |
| New pages slow to appear in Copilot | Missing from sitemap, no IndexNow | Add on publish; ping IndexNow |
| "Couldn't fetch" in Search Console | Wrong path or blocked | Check URL, robots.txt and status |
| Sitemap over 50,000 URLs | One giant file | Split and use an index |
How do you automate it?
Our view: generate the sitemap from the CMS on publish, update and delete events, not on a nightly timer that stamps every URL with the run time. Store a "last significant change" field separately from "last saved", so typo fixes don't reset lastmod. Bing suggests refreshing sitemaps at least once a day (Bing).
Exercise
Audit your sitemap with the six steps above. Record the share of URLs that fail each check, fix the generator, and re-submit in Search Console and Bing Webmaster Tools.
Checklist
- Canonical, indexable, 200 URLs only
- Honest lastmod with date and time
- No reliance on priority or changefreq
- Listed in robots.txt
- Submitted to Google and Bing
- IndexNow for instant updates
From MentionX data: which pages need to be discoverable
Method. Read-only aggregate queries over MentionX scans from 22 August to 29 October 2026 (UTC): 2,217 engine runs (2,154 completed) across 18 anonymised projects and six engines.
Sitemaps help crawlers find pages. Our sample shows which page types engines cite, and so which pages are most worth keeping in your sitemap with accurate dates:
| Cited URL type | Share of 9,750 citations |
|---|---|
| Blog or article | 19.6% |
| Homepage | 10.0% |
| Review page | 2.7% |
| Docs | 2.6% |
| Pricing | 1.9% |
| Wikipedia | 1.6% |
Our view: product, pricing, documentation and comparison pages are commonly cited and their facts change, so accurate lastmod dates on those pages matter most. A stale date on a pricing page can mean crawlers don't refetch it after a price change.
Limits. A convenience sample of mostly B2B software projects, run through engine APIs, which can behave differently from the consumer apps. Perplexity, Copilot and DeepSeek figures come from a single day (26 October 2026), so treat them as a snapshot, not a trend.
Worked example: Ledgerline fixes a stale sitemap
Ledgerline is a fictional accounts-payable tool. Figures are illustrative.
Ledgerline updates its pricing, but weeks later an AI answer still shows old prices. The team checks its sitemap: the pricing page's lastmod date is from last year, because the CMS only updates it when the page's title changes. Worse, the sitemap lists redirected and noindex URLs.
The team changes the CMS to set lastmod when the main content changes, removes non-canonical and non-200 URLs and splits the sitemap into sections for product, docs and blog. It resubmits in Search Console and Bing Webmaster Tools. Within days, logs show crawlers refetching the pricing page.
Which URLs should you leave out?
Leave out redirects, error pages, noindex pages, duplicate versions such as URLs with tracking parameters, and pages blocked by robots.txt. Including them sends mixed signals and wastes crawl attention. Every URL in a sitemap should be the canonical version, return 200 and be indexable.
How large can a sitemap be?
The sitemaps protocol limits each file to 50,000 URLs and 50 MB uncompressed. Larger sites use a sitemap index file that lists several sitemaps. Even if you're under the limit, splitting by section makes problems easier to spot.
Do changefreq and priority matter?
Google says it ignores changefreq and priority. Accurate lastmod is what counts. You can leave the other two out, which keeps files simpler.
How do sitemaps help AI search?
Several AI search features draw on search engine indexes. Google's AI features use Google's index, and Bing's index supports several assistants. Sitemaps help those indexes find and refresh your pages, so they help AI answers indirectly. Some AI crawlers may also read sitemaps, but vendors rarely document it, so don't rely on that.
Should you use IndexNow?
IndexNow lets you notify participating search engines, including Bing, as soon as a URL changes. Google doesn't use it. If Bing matters to you, and it supports several AI assistants, IndexNow is a cheap addition. Many CMS platforms and CDNs offer it as a setting.
How do you check sitemap health?
Use Search Console's sitemap report and Bing Webmaster Tools to see how many URLs were discovered and whether there are errors. Crawl the URLs in your sitemap with a site crawler to check status codes, canonicals and noindex tags. Compare the sitemap with your list of important pages to make sure nothing's missing. Do this monthly for large sites and quarterly for small ones.
What about image and video sitemaps?
Image and video sitemaps help search engines find media they might otherwise miss, such as images loaded by scripts. They matter mainly for image and video search. For AI answers, the text on the page matters more, so focus on the main sitemap first.
How do you handle pages that are removed?
Remove the URL from the sitemap the same day the page is removed or redirected. Leaving it in makes crawlers fetch a dead page and can delay them noticing the change. Our view: tie sitemap generation to your CMS publish and unpublish actions, so this happens automatically.
Which tools generate sitemaps?
Most CMS platforms generate sitemaps out of the box or with a common plugin, and many site frameworks have a sitemap package that builds the file at deploy time. Check the defaults: some include tag pages, author archives or attachment pages you don't want listed, and some set lastmod to the build date. Configure exclusions and the date source once, then spot-check the output after each upgrade. For custom sites, a small script that reads your published pages from the database and writes the XML is often simpler than a plugin, and easier to test.
Where should the sitemap live?
Put it at the site root, such as /sitemap.xml, and reference it in robots.txt with a Sitemap line. That way any crawler that reads robots.txt can find it, including ones you haven't submitted it to. A sitemap can only list URLs on its own host unless you verify cross-host ownership in each search engine's tools.
How do multilingual sites handle sitemaps?
You can list hreflang alternates inside the sitemap rather than in page HTML. This keeps page code lighter on large sites. Each URL entry lists all its language versions, including itself. Make sure every listed alternate is canonical and returns 200, and keep the sets complete.
What should a sitemap review cover after a migration?
After a domain move or URL restructure, publish a new sitemap with the new URLs on launch day. Keep the old sitemap briefly so crawlers can find redirects, then remove it. Check that the count of URLs roughly matches the number of pages you expect, and investigate any big gap.
Who should own the sitemap?
Engineering usually owns the generator, and SEO owns the rules about what's included. Agree both in writing. When a new section of the site launches, such as a help centre or a careers area, the launch checklist should include adding it to the sitemap index and confirming the URLs appear in Search Console within a week. Bing Webmaster Tools should be checked the same way. Check the sitemap again after every large release so new and removed pages are reflected quickly.
Common questions
Do AI crawlers read sitemaps? Some may, but vendors rarely document it. Sitemaps mainly help search indexes that AI features use.
Should lastmod change every day? No. Update it only when the main content changes.
How many sitemaps should I have? One index with separate files per section works well for most sites.
Related
Sources
- Google, build and submit a sitemap
- Bing, sitemaps in AI-powered search, July 2025; the importance of lastmod, 1 Feb 2023; webmaster guidelines
All accessed 30 Oct 2026.
Questions
Do changefreq and priority matter?
No. Google says it ignores priority and changefreq, and Bing says both are ignored and don't influence crawling or ranking.
How should lastmod be set?
To the real time of the last significant content change, in ISO 8601 with date and time. Not the time the sitemap was generated.
Do sitemaps help AI answers?
Bing says accurate lastmod helps updates reach search results and AI-generated answers faster, and that sitemaps support grounding eligibility in Bing and Copilot.