A site with several hundred pages without an XML sitemap is a concrete problem: recent content remains invisible for weeks because crawlers cannot find it. This situation occurs on e-commerce sites, on blogs with daily publications, or on institutional portals whose structure exceeds four levels of depth.
The sitemap does not solve everything, but it speeds up the discovery of URLs by Google and Bing by providing them with a structured list of what exists on the site.
The lastmod tag in an XML sitemap: a signal often misused
Most CMSs automatically fill in the lastmod tag of the sitemap with each page save, even for a cosmetic change. Google specifies that this tag is only useful if it reflects a significant update to the main content, structured data, or internal links. Modifying a footer or a copyright notice does not justify updating the date.
When the lastmod tag sends unreliable dates, Googlebot eventually ignores it. We then lose the only mechanism of the sitemap that allows signaling a recrawl priority. For a site that updates its product listings (prices, availability, descriptions), this is a wasted lever.
The best practice is to only fill in lastmod for changes that are truly observable: modified editorial content, new image added, updated structured data. If we cannot guarantee the accuracy of the date, it is better to omit the tag than to send false signals. A concrete example can be checked by consulting the sitemap of the Experts Marketing site, which illustrates the structure of an XML file with its date tags.
Sitemap and crawl budget: why file size matters
On a site with a few dozen pages and a good internal linking structure, the XML sitemap is almost redundant. Google will find the URLs by following the links. The situation changes radically for large sites or those where some pages are poorly linked.
The sitemap protocol imposes a limit of 50,000 URLs per file and a maximum uncompressed weight defined by the standard. Beyond that, it is split into several files referenced by a sitemap index. This splitting is not just a technical formality: an overloaded sitemap with obsolete URLs dilutes the crawlers’ attention on the important pages.
- Exclude redirected URLs (301), noindex pages, and canonicalized URLs pointing to other pages from the sitemap. Their presence generates errors in Google Search Console and muddles the diagnosis.
- Separate content types (articles, product sheets, images, videos) into distinct sitemap files for easier tracking in Search Console.
- Automatically regenerate the sitemap with each publication or deletion of content, rather than at fixed intervals, to avoid discrepancies between the file and the reality of the site.
Feedback varies on the direct impact of the sitemap on crawl budget: Google considers the sitemap a discovery signal, not a directive. But for catalogs with several thousand references, a clean sitemap remains the most reliable way to signal new URLs without waiting for an internal link to be manually created.
Submission in Google Search Console: what the sitemap report really reveals
Submitting a sitemap via Search Console does not guarantee the indexing of the listed URLs. This is a point that many guides overlook. The sitemap is a discovery signal, not an indexing instruction. Google then decides, page by page, whether the content deserves to be indexed.
The sitemap report in Search Console indicates the date of the last access by Googlebot and the number of detected URLs. This diagnosis allows for quick identification of a problem:
- If the number of discovered URLs is lower than the number of submitted URLs, some lines in the file contain errors (URLs blocked by robots.txt, 404 responses, looping redirects).
- If Googlebot has not accessed the sitemap for several weeks, the file may not be accessible or the server is returning an intermittent error code.
- If indexed URLs are not present in the sitemap, it means the file is incomplete or that the automatic regeneration is malfunctioning.
This report is used as a monitoring tool, not as a dashboard for overall SEO. Cross-referencing sitemap data with the index coverage report provides a more complete view of the site’s crawl status.
XML or HTML sitemap: choose based on real needs
The XML sitemap is aimed at crawlers. The HTML sitemap is aimed at visitors. The two have neither the same format nor the same utility, and the confusion between the two persists.
For natural SEO, the XML file remains the priority. It structures the URLs with their metadata (modification date, suggested change frequency) in a format readable by search engines. The HTML sitemap, on the other hand, takes the form of a standard web page that lists the site’s sections with clickable links.
An HTML sitemap can improve navigation on a site with a complex architecture, but it does not replace the XML file for content discovery by search engines. On a well-structured site with a solid internal linking, the HTML sitemap does not add much. On an institutional site with dozens of poorly linked sub-sections, it helps visitors find a page without going through the internal search.
The choice depends on the site’s profile. A blog with fewer than fifty articles and clear navigation can rely on the native XML generated by its CMS. An e-commerce site with deep categories and faceted filters needs a segmented XML sitemap and, possibly, an HTML sitemap for its users.
The sitemap remains a technical file among other optimization levers. Its real contribution is measured in Search Console, by observing index coverage and the speed of discovery of new pages. A clean, up-to-date file, purged of unnecessary URLs, saves time for crawlers and those diagnosing indexing issues.
