SEOmator's free sitemap finder reads a site's robots.txt and the standard sitemap paths, then reports every file it finds with its HTTP status, format and URL count — expanding sitemap index files one level, up to 50 files per run. Of the top-1,000 sites we could reach on 1 October 2026, only 52.9% exposed a valid sitemap. No signup.
Reviewed by Charles Duncan, Co-founder & CTOThe checker does not guess at URLs. It reads the site's robots.txt for declared Sitemap: lines first — the same place a crawler looks — then falls back to the two conventional paths, fetches each candidate directly, and follows any sitemap index one level down to its children. Every file it reaches is reported at the URL it finally resolved to after redirects, so a sitemap quietly redirecting elsewhere is visible rather than counted as working.
Paste a domain or a full page URL. The checker reduces it to the site's origin, because sitemaps are declared at the host level rather than per page.
robots.txt is read for declared Sitemap: lines, then /sitemap.xml and /sitemap_index.xml are tried as fallbacks. Duplicates are collapsed, and a sitemap index is expanded one level to its child files, up to 50 in total.
Each file returns its final URL after redirects, its HTTP status, the content-type it was actually served with, and how many entries it holds.
Everything below runs without an account, in a single pass. A free SEOmator account adds the parts that need to remember yesterday.
Reads the site's robots.txt for declared Sitemap: lines before falling back to /sitemap.xml and /sitemap_index.xml — the same order a crawler works in.
When a file is a sitemap index, the checker follows its child sitemaps one level down and reports each one on its own row, up to 50 files per run.
The real response code per sitemap: a 200, a redirect chain, a 404, or a fetch that failed outright. A declared sitemap returning 404 is the most common finding there is.
Counts the entries in each file, so you can see whether a sitemap holds 12 URLs or 48,000 — and whether that matches the site you expected to find.
Counts <loc> entries in XML, <link> items in RSS feeds, and links in HTML sitemaps, and reports the content-type each file was genuinely served with.
Shows where each sitemap URL actually landed. A sitemap that 301s to the homepage is a silent failure the status code alone would not reveal.
Track how a sitemap's URL count moves week to week, so a sudden collapse is caught before it costs you indexing.
Get notified when a sitemap starts returning a non-200 or empties out, instead of discovering it at the next manual check.
A full site audit fetches the URLs inside the sitemap and reports the ones that 404, redirect, or carry a noindex tag.
Create a free SEOmator account to unlock sitemap history, break alerts, and full site audits.
Sign up free
Reviewed byCharles Duncan · Co-founder & CTOI built this to answer the question a crawler asks first: does this site declare a sitemap, and does the file actually come back? Most sitemap problems I see are not malformed XML — they are a Sitemap: line in robots.txt pointing at a URL that 404s, or an index whose child files were never deployed, and both surface here on the first run. The honest limit is discovery: we read robots.txt and the two conventional paths, so a sitemap sitting at an undeclared custom path is one this tool will not find, and the site may be perfectly healthy anyway. It also counts entries rather than validating them — a sitemap can return 200 with 40,000 URLs in it and every one of those URLs can redirect, which is a job for a crawl, not a lookup.
Charles Duncan · Co-founder & CTOThe Sitemap: directive is the discovery path you control that every crawler reads. Of the top-1,000 sites we could reach on 1 October 2026, only 50.4% declared a sitemap in robots.txt, and 7.0% had a working one at a default path that robots.txt never mentioned. Declaring it costs one line.
A declared sitemap that 404s spends the request a crawler allocated to discovery and returns nothing for it. If you are retiring a sitemap, remove its robots.txt line in the same deploy.
The count column is the fastest sanity check available. A 4,000-page shop whose sitemap declares 61 URLs has a generation bug, and a 20-page brochure site declaring 900 is publishing filtered or paginated URLs it did not mean to.
A sitemap index that returns 200 tells you nothing about its children. This tool expands one level; if your index points at further indexes, check the deepest files directly — that is where empty or 404 sitemaps hide.
Google has said plainly for years that it ignores both, and neither has ever influenced crawl scheduling. The element worth getting right is lastmod, which Google does use — but only from sites where it has proven honest.
Platform moves, CDN changes and CMS upgrades are where sitemaps break, usually by starting to redirect or by regenerating at a new path while robots.txt still names the old one. Check on deploy day, not at the next audit.
There are three places a sitemap can be announced, and they are worth trying in this order — it is the order a crawler itself works in.
example.com/robots.txt and look for a line beginning Sitemap:. This is the only discovery mechanism defined by the sitemaps protocol itself, it can appear anywhere in the file, and a site may declare several. It is also the only one that finds a sitemap published at a non-obvious path.example.com/sitemap.xml and example.com/sitemap_index.xml. Nothing in the spec requires either location, but almost every CMS and SEO plugin writes to one of them, so the convention holds most of the time.<sitemapindex> rather than a <urlset>, it contains no page URLs at all — only links to further sitemap files. The real content is one level down, and that is where problems tend to hide.The checker above does all three in one pass and expands any index it finds. What it will not do is guess: a sitemap at a custom path that robots.txt never declares is invisible to it, and to most crawlers too — which is precisely why the robots.txt line matters more than the file's location.
Searching Google for site:example.com filetype:xml occasionally surfaces a sitemap that is indexed but undeclared. Treat that as a diagnostic curiosity rather than a method; if a sitemap is only findable that way, the fix is to declare it.
A sitemap report is four facts per file, and each one fails in its own characteristic way.
The final URL is where the request actually landed, not what you asked for. If you requested /sitemap.xml and the report shows /sitemap_index.xml, the site is redirecting — usually harmlessly. If it shows the homepage, the sitemap does not exist and the server is papering over the 404 with a redirect, which is considerably worse than a clean 404 because nothing in the status code says anything is wrong.
The status is the response code after those redirects resolve. A 200 means the file came back. A 404 on a sitemap that robots.txt declares is the most common single finding in sitemap checks, and it is a self-inflicted one: the line outlived the file. A status of 0 means the fetch failed entirely — DNS, TLS, or a timeout — which points at infrastructure rather than at the sitemap.
The content type is what the server said it was sending. XML sitemaps should arrive as application/xml or text/xml. Seeing text/html on a URL ending in .xml almost always means you are looking at an error page dressed as a success, and the entry count in that row will be meaningless. Gzipped sitemaps arrive as application/x-gzip and are entirely valid — search engines have accepted them for years. One honest limit of this checker: it does not yet decompress raw gzipped files, so a .xml.gz sitemap currently shows 0 entries even when it is full. Open it in a browser or decompress it locally to count it.
The entry count is how many URLs the file declares. This is the number worth checking hardest, because it is the one that silently drifts. A sitemap is not required to be complete, and nothing warns you when it stops being so.
Most sitemap problems are not malformed XML. Of the 793 sitemap URLs that top-1,000 sites declared in robots.txt on 1 October 2026, 87.1% returned a valid sitemap and 5.3% were genuinely broken, mostly 404s and HTML pages served in place of the file; broken markup was the rare case. What actually goes wrong is more mundane:
noindex, or are blocked in robots.txt, or canonicalise elsewhere. The sitemap says "index this," the page says "do not." The page wins, and the sitemap entry is wasted.The last two are worth dwelling on, because a sitemap checker cannot see them. Discovery confirms the file exists and counts what it declares; only fetching each of those URLs reveals whether they resolve, redirect, or refuse indexing. That is a crawl — SEOmator's free SEO audit does it — and it is the natural next step after the sitemap itself checks out.
The sitemaps protocol defines four elements per URL. Search engines stopped treating two of them as meaningful a long time ago, and a great deal of published advice has not caught up.
| Element | Does Google use it? |
|---|---|
<loc> | Yes — the URL |
<lastmod> | Yes, when trusted |
<changefreq> | No |
<priority> | No |
<changefreq> and <priority> are ignored. Google has stated this repeatedly and plainly; neither has ever influenced how often a page is crawled or how it ranks. Setting every page to priority 1.0 does nothing, and setting them thoughtfully does nothing either. Time spent tuning these fields is time spent on nothing at all.
<lastmod> is different — Google does use it, but conditionally. It is treated as a hint whose weight depends on whether the site has proven honest with it. A site that stamps every URL with today's date on every regeneration teaches crawlers to disregard the field entirely, and that trust is not scoped to the URLs you lied about. Either set lastmod to the date the page's content genuinely changed, or omit it.
The practical implication is narrow and useful: the only thing a sitemap really does is tell a search engine that a URL exists and, sometimes, when it last changed. Everything else in the file is decoration.
The sitemap has quietly become more useful than it was, because a new category of crawler depends on it more heavily than Googlebot does.
Of the Tranco top-1,000 sites we could reach on 1 October 2026 (675 of 1,000), 52.9% exposed a valid XML sitemap through robots.txt or the default /sitemap.xml and /sitemap_index.xml paths. Only 50.4% declared one in robots.txt, the first place a crawler that has never seen the site looks. Even among the web's largest sites, a crawler cannot count on being handed a list of pages.
That gap matters because AI crawlers generally have less patience than search engines. Googlebot has spent two decades building a link graph of the web and can find pages by following links from sites it already trusts. A retrieval crawler fetching pages to answer a question has no such graph, a much smaller budget, and far less tolerance for deep link chains. A declared sitemap is the cheapest way to hand it a complete list instead of asking it to explore.
This does not require a separate AI sitemap, and no such format exists. The ordinary XML sitemap, declared in robots.txt, accurate, and returning 200, is the whole requirement — and the same file that helps Googlebot helps everything else. Check the robots.txt side of that with the free robots.txt tester.
A sitemap is a discovery aid. It has no influence on ranking, and it cannot compensate for the problems that actually keep pages out of an index.
If a page is not indexed, a sitemap entry will not fix it when the page is thin or duplicative, when it carries a noindex directive, when it canonicalises to a different URL, when robots.txt blocks the path, or when it requires JavaScript that a crawler does not execute to render its content. In each of those cases the crawler found the page perfectly well and declined it. Adding it to a sitemap declares it more loudly to a crawler that already made its decision.
Small, well-linked sites often need no sitemap at all. If every page is reachable in two or three clicks from the homepage, ordinary crawling finds everything, and the sitemap is a formality. The sites where it genuinely changes outcomes are large ones, sites with deep archives, sites with weak internal linking, and new sites with almost no inbound links — the cases where discovery is the actual bottleneck.
The honest summary: a sitemap is a cheap, high-leverage fix when discovery is broken, and irrelevant when it is not. Checking that it resolves and declares roughly the right number of URLs takes seconds. Whether those URLs are worth indexing is a separate question, and a harder one.
Check a prospect's sitemap before the first call. A declared sitemap that 404s, or a count that does not match the site's real size, is a concrete finding you can open with rather than a generic audit promise.
Verify after every deploy that the sitemap still resolves, still returns 200, and still declares roughly the number of URLs it did yesterday — the three things a release most often breaks silently.
Confirm what your SEO plugin actually publishes. Plugins frequently generate an index plus several child files at paths nobody on the team has ever opened, and the counts are where mis-configuration shows up.
Most sitemap checkers stop at the first file and call an index a success. This one follows the index down to its children and reports each of them separately.
Discovery starts at declared Sitemap: directives rather than at a guessed /sitemap.xml, so it finds sitemaps published at non-default paths — and shows when robots.txt names one that no longer exists.
Each row shows the URL the request finally landed on. A sitemap silently redirecting to the homepage looks like a working sitemap in tools that only report a status code.
XML, RSS feeds and HTML sitemaps each store their entries differently. The counter matches the format rather than assuming every file is XML.
The full report — discovery, expansion, statuses and counts for up to 50 files — runs without an account, an email address, or a trial.
Sitemaps are one half of crawl discovery. Check the other half with the free robots.txt tester, then run a free SEO audit to crawl what the sitemap declares.
A sitemap is a file that lists the URLs on a website so search engines can discover them without relying entirely on following links. It does not make pages rank, but it makes them findable — which matters most on large sites, new sites with few backlinks, and sites where some pages are not well linked internally.
No. Of the Tranco top-1,000 sites we could reach on 1 October 2026, 52.9% exposed a valid XML sitemap through robots.txt or the default paths, so even among the web's biggest sites a sitemap is far from universal. A small, well-linked site can be crawled fully without one; a large or poorly linked site usually cannot.
Check robots.txt first — open example.com/robots.txt and look for a line beginning Sitemap:, which is where sites are supposed to declare it. If there is none, try example.com/sitemap.xml and example.com/sitemap_index.xml. The checker above does all three at once and also expands any sitemap index it finds.
A sitemap is plain XML, so any browser will display it — paste the URL into the address bar. Large sitemaps are often served gzipped as .xml.gz, which downloads instead of displaying; unzip it and open the .xml file in a text editor. A sitemap index looks similar but lists other sitemaps rather than pages.
Every page URL in an XML sitemap sits inside a <loc> tag, so extracting them is a matter of pulling that element out — most editors will do it with a search, and command-line users can pipe the file through grep or an XML parser. Start with the index if there is one, since the page URLs live in the child files rather than the index itself.
Anywhere on the same host, as long as robots.txt declares it. The convention is the root — example.com/sitemap.xml — and it is worth following because some tools only check there, but a sitemap at a custom path is entirely valid when the Sitemap: line in robots.txt points to it.
Whenever the set of URLs changes. Most CMS platforms and SEO plugins regenerate the sitemap automatically on publish, so the practical task is verifying that it happened rather than doing it by hand — particularly after a migration, when regeneration often starts writing to a new path.
Declare it in robots.txt with a Sitemap: line, which every major crawler reads, and submit it in Google Search Console and Bing Webmaster Tools. Submission is a one-time action: once a sitemap URL is known, crawlers re-fetch it on their own schedule and there is nothing to resubmit after each update.
An XML sitemap is a machine-readable file listing URLs for search engines, and it is the one that matters for crawling. An HTML sitemap is an ordinary page listing links for human visitors. They serve different audiences, and having one does not replace the other — though for most sites the XML sitemap is the one worth maintaining carefully.
Deeper reading from the SEOmator blog on what this tool measures.
![How to Find the Sitemap of a Website [8 Ways]](/_next/image?url=%2Fimages%2Fblog%2F223df5916e2af0cf.webp&w=2048&q=75&dpl=dpl_GqyqGp9VBb7agKcnT7bnWBuktCDv)
Just like a blueprint that outlines a house's design, a sitemap helps search engines navigate a site efficiently and index all pages properly. In this sense, sitemaps play a pivotal role in any website's SEO strategy.
We checked the top 1,000 domains: 1 in 5 robots.txt files now fully blocks an AI crawler. The syntax, the rules Google ignores, and how to handle GPTBot.
Cloudflare Radar: Perplexity now crawls 885 pages per visitor it sends back and Anthropic 823 - down by half. Plus SEOmator's own panel data and 30+ AI SEO statistics.
The checker confirms your sitemaps resolve and how many URLs they declare. SEOmator's free SEO audit crawls those URLs and reports the ones that 404, redirect, or block indexing.
