Sitemaps “Ping” Endpoint Going Away

The Sitemaps Protocol has been a valuable tool for search engines since its introduction in 2005. It has facilitated the discovery of new URLs and the scheduling of crawls for existing ones. However, as the internet landscape has evolved, some protocol aspects have become less practical. In light of this, we are announcing the deprecation of the sitemap “ping” endpoint and providing updated recommendations for using the “lastmod” element

Sitemap Ping Deprecation

The sitemap protocol currently allows for unauthenticated REST method submissions to search engines via the “ping” endpoint. However, our internal studies and insights from other search engines like Bing indicate that these unauthenticated sitemap submissions are less beneficial. Unfortunately, the majority of these submissions have been associated with spam. As a result, we are deprecating support for sitemaps ping, and the endpoint will cease to function in the next six months. While you can submit sitemaps through robots.txt and Search Console, HTTP requests (or “pings”) to the deprecated REST endpoint will return a 404 error. Any existing code or plugins utilizing this endpoint will not affect Google Search, but it will not have any meaningful impact.

The Lastmod Element

Over the years, we have noticed varying degrees of usefulness of the “lastmod” element across different sites. This might be due to the type of content published or the content management system used. However, the “lastmod” element has proven valuable in many cases, and we now use it as a signal to schedule crawls for URLs we have already discovered.

For the “lastmod” element to be effective, it must be in a supported date format (documented on sitemaps.org). Search Console will notify you if it’s not in the correct format when you submit your sitemap. Additionally, the “lastmod” value must consistently align with reality. If a page’s content changed seven years ago, but the “lastmod” element states it changed yesterday, it will eventually lose credibility as a reliable indicator of its last modification date.

Using the Lastmod Element

You can include a “lastmod” element for all pages in your sitemap or only for the ones you are confident about. For example, if your site software aggregates other pages on the site, you may find it challenging to determine the last modification date for the homepage or category page. In such cases, omitting the “lastmod” element for those pages is acceptable. Here’s an example:

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
      <lastmod>2005-01-01</lastmod>
      <loc>http://www.example.com/</loc>
      <changefreq>monthly</changefreq>
      <priority>0.8</priority>
  </url>
</urlset>

Significant Modifications

When we refer to “last modification,” we specifically mean “last significant modification.” If your content management system made an insignificant change to the sidebar or footer, there is no need to update the “lastmod” value for that page. However, if you made significant changes to the primary text, added or modified structured data, or updated important links, updating the “lastmod” value is recommended.

Takeway(s)

As we bid farewell to the sitemaps “ping” endpoint, we encourage you to use the “lastmod” element to optimize crawls for URLs on your site. By following the best practices and recommendations provided, you can continue to ensure that search engines effectively discover and crawl your content. We will make product announcements about new features that further improve indexation.