zh
English Español Deutsch Français Italiano Português (Brasil) Русский 中文 日本語
提交帖子
前往博客
Kevin Indig 已更新

XML Sitemaps Guide: The Best Tricks, Tips & Sitemap Generator Tools

47
Wow-Score
Wow-Score 表示博客帖子的参与度,是根据用户的主动阅读时间、滚动速度和文章长度之间的相关度计算得出。
此帖子为 English
Kevin Indig 已更新
此帖子为 English
XML Sitemaps Guide: The Best Tricks, Tips & Sitemap Generator Tools

In my first weeks at Atlassian, I was seeking for a quick-win to get a stronger foothold within the company and evangelize SEO. So, I audited our main site and noticed it didn’t have an XML sitemap. What an easy win!

I went to the devs and asked them to activate it in the CMS. To my surprise, they told me that it wasn’t possible; I was baffled.

After some thinking, I remembered that Screaming Frog had an XML sitemap function, so I scraped the site and uploaded the crawl as an XML sitemap.

Google ate it within a few seconds, and we saw a noticeable impact on our traffic in the following days.

The moral of the story is that XML sitemaps are important and sometimes underrated.

What XML Sitemaps Are and Why You Need to Have One

XML sitemaps are digital maps that help Google discover important pages on your site and how often they are being updated.

Google states on its help center page:

A sitemap tells the crawler which files you think are important in your site, and also provides valuable information about these files: for example, for pages, when the page was last updated, how often the page is changed, and any alternate language versions of a page.

According to Gary Illyes, XML sitemaps are the second most important source of URLs to be crawled by Googlebot after hyperlinks and previously discovered URLs. That’s massive and shouldn’t be underestimated!

Every website should have an XML sitemap. They are especially important for:

  • Large sites

  • New sites

  • Sites with lots of orphaned pages

  • Sites that use lots of images and videos

Whereas the robots.txt helps you to exclude parts of your site from being ranked in search engines, XML sitemaps do the opposite. They help search engines discover new pages – even when they are not linked from the main site.

Google started using XML sitemaps in 2005 and shortly after was joined by search engines like MSN or Yahoo. Nowadays, they use them for even more than just URL discovery.

Don’t confuse them with HTML sitemaps. XML sitemaps are not visible to visitors, only to Google and the SEOs that want to look at important pages of competitors (wink wink). HTML sitemaps are (sometimes pretty) lists of internal links that pass PageRank from the homepage to important pages, serve as crawl hubs, and make it easier for users to navigate. Yes, you should have both.

Sitemaps come in XML format that Google can quickly parse to find new URLs. XML — eXtensible Markup Language — is lightweight and portable between devices and was made to store data.

The easiest way for you to check if your site has a sitemap is to look in Google Search Console or in Bing Webmaster Tools under “sitemaps.” Most search engines, such as Google or Bing, look for the “Sitemap: <sitemap_location>” entry (or entries) in your site’s robots.txt file. Alternatively, you can also ping your sitemap directly to Google, Baidu, Bing, and Yandex.

xml sitemap in Bing Webmaster Tools XML sitemaps in the Bing Webmaster Tools. 1: Sitemaps report. 2: Adding new sitemap paths. 3: existing sitemaps Bing found.

XML sitemap in Google Search Console XML sitemaps in Google Search Console. 1: Sitemaps report. 2: Adding new sitemap paths. 3: existing sitemaps Google found.

XML Sitemap Minimum Requirements

For your XML sitemaps to work optimally, you have to meet the standards. An XML sitemap should:

  • Contain only canonical URLs with a 200 status code.

  • Include up to 200K URLs per sitemap and up to 50K sitemaps per index sitemap.

  • Be referenced in the robots.txt.

  • BUTF-8 encoded.

  • Be compressed in .gz format.

  • Be no larger than 50mb or contain 50,000 URLs (whatever you hit first).

But there is more you can and should do to get the most out of XML sitemaps. You can signal to Google which URLs are important by including only important pages in XML sitemaps, and by updating it often.

Most CMSs have a function to automatically update sitemaps when a new URL is created or an existing page changes. For Google, the update frequency of the sitemap itself and the lastmod tag of pages can be a signal of freshness. Whether that is important for its ranking depends on the page and the context.

Think of an XML sitemap as a city map for tourists with the city being your website and the tourist being Google — you want to make sure to only include the important buildings, not every address. That is why unimportant pages shouldn’t be included; examples are pages like your privacy policy or about us page. While these pages should be indexed, they don’t need to be crawled often and are not important when we are talking about SEO.

XML Sitemap Tips for Large Sites

There is more you can do to elevate your sitemap game, beyond meeting the standard requirements.

Large sites like news publishers, for example, should make use of index sitemaps, which contain (up to 50,000) normal sitemaps, and should also not be heavier than 50mb. They are like the XML sitemap mothership that carries lots of smaller sitemaps. Large sites need them because they can’t fit into a single sitemap. You shouldn’t try to fit everything into a single sitemap, anyway.

You can make the most out of these sitemaps by structuring them either per page type or topic. In practice, you would create dedicated XML sitemaps per subdirectory or page template to get an understanding of technical and indexing problems with your site.

GSC submitted sitemap xml

There are specialized XML sitemaps for specific purposes. Sites that operate heavily around rich media (think: Pinterest or YouTube) benefit a lot from image or video sitemaps. Publishers should have news sitemaps.

Image sitemaps increase your site’s chance to be found in Google image search. You don’t have to have a dedicated image sitemap; you can also use image extensions in your regular sitemap.

This is what image extensions look like (XML specifications):

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:image="http://www.google.com/schemas/sitemap-image/1.1">
<url>
<loc>http://example.com/sample.html</loc>
<image:image>
<image:loc>http://example.com/image.jpg</image:loc>
</image:image>
<image:image>
<image:loc>http://example.com/photo.jpg</image:loc>
</image:image>
</url>
</urlset>

Video sitemaps function after the same principle: either create a dedicated sitemap or add extensions to your regular one:

<url>
<loc>https://example.com/mypage</loc>
<video> ... information about video 1 ... </video>
</url>

But be careful with the meta-data you add to video sitemaps or extensions.

Google states, “Google might use text on the video landing page rather than the text you supply in your sitemap if the page text is deemed more useful than the information in the sitemap.” They are speaking about the text delivered through the description. Besides a description, you can feed Google a thumbnail, video length, rating, family-friendliness, and more (full list of video XML sitemap meta-data). For sites that heavily use video, this certainly makes sense. For all others, it is relatively optional.

News sitemaps are different in that you should always have a separate news XML sitemap. Google doesn’t recommend (or offer) extensions in this case. News sitemaps help Google discover and rank new articles, which is especially challenging in the publishing industry because it produces a lot of content. Even though Google states that publishers with news sitemaps are not favored, it does help to get hot news ranking in Google News faster.

News sitemaps have special requirements:

  • Include articles not older than 2 days.

  • Don’t add more than 1000 new entries to an existing sitemap at a time.

  • Update existing sitemaps for article updates.

You can also use XML sitemaps to define and indicate certain meta-tags for Google. One example is hreflang; you can add as an extension to a sitemap (full guidelines):

<url>
<loc>http://www.example.com/english/page.html</loc>
<xhtml:link
 rel="alternate"
 hreflang="de"
 href="http://www.example.com/deutsch/page.html"/>
<xhtml:link
 rel="alternate"
 hreflang="de-ch"
 href="http://www.example.com/schweiz-deutsch/page.html"/>
<xhtml:link
 rel="alternate"
 hreflang="en"
 href="http://www.example.com/english/page.html"/>
</url>

Google ignores the priority attribute in XML sitemaps but does pay attention to lastmod, according to John Mueller. Google determines the priority of your pages itself, probably by popularity and authority. Lastmod, however, is a tag that indicates when the URL has changed the last time, which is really interesting to Google.

Also, you don’t need to add XML sitemaps for AMP URLs, according to John Mueller.

How Do Web Crawlers Scan Your Site?

A new tool to understand how search engine bots work

Please specify a valid domain, e.g., www.example.com

XML Sitemap Tools and Generators

Most content management systems come with prepackaged functions that allow you to create an XML sitemap automatically. But some don’t, and in this case, you need a third-party tool.

Best Sitemap Generators

These are my personal picks for XML sitemap generators.

Name

Price

Limit

Features

Free trial

Slickplan

$8.99/month

n/a

  • Drag and drop builder
  • Custom page type inclusion
  • Import text file
  • Cloning
  • Batch editing
  • Highly customizable
  • User permissions
  • Custom branding

30 days

Dynomapper

$40/month

200K URLs per crawl

  • Monitor URLs in sitemaps in Google Analytics
  • Highly customizable
  • Custom page type inclusion
  • Workflow management
  • URL tagging
  • Sitemap filtering
  • User permissions
  • Custom branding

14 days

Writemaps

$14.99/month

n/a

  • Customizable
  • Custom groups
  • Drag and drop builder

3 sitemaps free

Screaming Frog

Free to 500 URLs

£149.00/year

n/a

  • Not made for XML sitemaps but good workaround for technical restrictions

n/a

Powermapper

$49/month

15K pages

  • Not made for XML sitemaps but can export a feed into XML format

30 days

XML Sitemaps

$4.99 for 1K pages

$189.99 for 1.5m pages

1,5m pages

  • Image and video sitemaps
  • Email notifications
  • Mobile app
  • Detects broken links

Free for 500 pages

Best Wordpress Plugins for Sitemaps

Name

Price

Limit

Features

Ratings

Simple Wp Sitemap

free

n/a

  • HTML and XML sitemap
  • Dynamic sitemaps

4.4/5

33 reviews

Google Sitemap by BestWebSoft

free

n/a

  • Hreflang support
  • Customizable

4.3/5

112

Google XML Sitemaps

free

n/a

  • Dynamic sitemaps
  • Customizable

4.9/5

2090 reviews

Yoast SEO

free (premium available)

n/a

  • Basic, dynamic sitemap

4.9/5

26,745 reviews

WordPress XML Sitemap Plugin

free

n/a

  • Highly customizable

n/a

All in One SEO Pack

free

n/a

  • Basic, dynamic sitemap

4.4/5

449 reviews

XML Sitemap & Google News

free

n/a

  • Basic, dynamic sitemap
  • Customizable
  • Updates lastmod automatically

4.4/5

59 reviews

Do you have any XML sitemap tips or tools? Let us know in the comments.

Get a free 7-day trial

Start working on your online visibility

Please specify a valid domain, e.g., www.example.com

Kevin Indig
专栏作家

SEMrush 专栏作家是指定期为我们的社区做出贡献的数字营销资深作者。

发送反馈

您的反馈必须包含至少 3 个单词(10 个字符)。

我们只会使用您的电子邮件,以回答您的反馈。 隐私政策

感谢您的反馈!