English Español Deutsch Français Italiano Português (Brasil) Русский 中文 日本語

SEMrush ToolBox #2: Site Audit tool


Please give us your feedback.



Craig: Hi guys, and welcome to today's SEMrush Webinar. It's the SEO toolbox part 2. Today I'm going to be joined by Dawn Anderson, who I'm sure needs no introduction to you guys. She's been on SEMrush plenty of times.

On today's webinar, we are going to be talking about the Site Audit tool, and it will be similar to the last SEO toolbox webinar. I'll give just a brief demonstration of the tool as an overview. Then we're going to have Dawn give her expert advice on things that she comes across when auditing websites and things that Dawn feels is really important, and various other bits and bobs too. We will also have questions and answers.

So for anyone who doesn't know of you Dawn, do you want to tell the guys a bit about what you do?

Dawn: So I'm an SEO consultant primarily, but I also consult on overall web strategy, digital strategy. I lecture on digital strategy at the university and I supervise master's level students that are doing their dissertation in digital marketing strategy.

SEMrush Site Audit Tool Overview

Craig: I think we will press ahead and get the show on the road. So what I'll do is I'll just start sharing my screen. I will talk you guys through the Site Audit tool. So obviously you go into projects, set up your project, do a site audit. What you'll be then faced with is a dashboard. So this is just an example website, Tesco.com, that I've done an audit on.

I find it astonishing the number of websites that I open that have glaringly obvious errors totally missed. Internal links are broken, duplicate stuff, everything else there. When you go into SEMrush, you've got this full interface here. One of the first things you're going to see is errors, which you can click on, and it'll bring up all of the errors that you've got on there.

So that, if you click on 46 pages with duplicate meta descriptions, it will show you the pages that have the duplicate meta descriptions. You do also have advanced filters here. So you can filter all this kind of stuff down, you can filter by page URL. You can send these errors to Trello.

So for example, if you do site audits but you're not the person that does the changes on a website, you can send these to Trello and make good use of that function within the tool. A lot of people do ignore the fact that you can just send all that information straight over to Trello without too much bother at all. So that is one of the first options there, but I'm just going to go through these tabs first and foremost.

So overview gives you your overview of errors, warnings, and notices. Issues are going to replicate that stuff, and it just breaks it down to errors, warning, and notices. Crawled pages; it's going to show you the pages, the site structure of your website, and it's also got the internal links. You can use various other options here as well, sitemap, canonicalization, markup, and various other bits and bobs.

Statistics, it's more of a kind of graphy, kind of stats thing. I don't personally do too much with it, but it will show you various other statistics based around your website.

Twitter Cards is there, and that's something you can relay back to a customer because when you're doing technical SEO, and justifying yourself to a customer, saying that you've got this or you've got that isn't always enough to make it sound as if you're doing a lot of work. You can actually show them these graphs and stuff like that as well, which I think sometimes helps hit home in a customer's head more than anything. So showing them these statistics is sometimes useful.

You can also compare the crawls. So I've only done this crawl today prior to the webinar, but if I crawled this website just now, then I crawl it next week and then the following week, then I can compare the data and see that I'm getting improvements.  

That's something else that the progress tab will have as well. So you'll be able to see the progress and improvements here. It's telling me we've got 8,339 issues. Hopefully, if I was to send that to Trello and get someone to action some of the issues, you would hopefully see that graph go down.

So that's the basics of the top tab there, but going into everything else, as I say, errors which I looked at earlier on, which is just all your errors. The middle one is warnings. So errors are things that potentially are broken and need fixing, probably with some form of urgency. Warnings are things like low word counts, temporary redirects, uncompressed stuff.

Again, notices, I mean, some of these things still need to be done; it will tell you 38 pages are blocked from crawling, but you may be deliberately doing that for other reasons which Dawn may talk about later on.

So notices are more of a, "You might want to look at this," but warnings are something that you see a hell of a lot of. That (warning) could be you've not installed your SSL certificate properly, or various other things. As I say, it depends on the website.

Other than that, there are thematic reports here, such as crawlability. You just click on these and view the details. It will tell you that this particular website has 343 non-indexable pages. That's going to instantly flag up a problem in my head going, "Why can these pages not be indexed?"

You've got this part here where it says your crawl budget is being wasted on certain things such as temporary redirects, duplicate content, pages being blocked from crawling, various other things. Obviously what you want to do with your website's crawl budget is make sure that your crawl budget is used on pages that are there to rank.

So again, SEMrush's audit tool does go in-depth. It's more than telling you that you've got missing meta descriptions or whatever it may be. So there is a lot more to this tool, and that's what the crawlability part does there.

HTTPS; it will tell you what your score is in terms of HTTPS implementation. We've got 95% here. It is telling you here 1,472 links on HTTPS pages leads to an HTTP page. So you want to get that looked at and make sure that it's 100% if you can get that. So make sure that your SSL certificate is installed properly.

International SEO, again, I'm not going to touch too much on that because that's more Dawn's bag than mine, but you can look at that within SEMrush on the audit tool as well. You can see here that there are five pages with issues, one page without issues. You'd have to dig deep and see what those issues are.

Site performance. Again, site speed is massively important. It is allegedly a ranking factor and something that I think a lot of people are now becoming more and more obsessed with. SEMrush do allow you to get a good look at the speed of your website and stuff like that here, and you can see certain issues using this particular tab. So the score's 86%, so it's not outrageously bad, but we always want to aim for the highest score possible.

Internal linking. Again, there are people out there who do some great internal linking and get a massive uplift on their website. So doing that one single thing can have a massive impact on your overall rankings. It will tell you about broken internal links and various other parts, crawl depth and various other stuff on there that is really important.

I think people that I speak to out there just think that SEMrush just does a quick site audit and boom, that's it. It chucks out some report. It does a lot more than that. Overall, it's a great go-to tool. It's one of the first tools that I run things through. It will flag up errors. It's not because it's SEMrush that I go to it for anyone who thinks because I'm in a webinar I'm going to say that. Certainly not. I think that the tool is very aggressive and very in depth, and it is one of the better ones out there that flags up stuff.

Obviously, I will use various other tools, digging deeper into certain things. But I think as an all-in-one tool, this just allows you to overview something, pass it on to one of your team, whether that's through Trello, or export the PDF. So you can pull all this data over and give it to a customer as well.

So if for example, you're faced with a customer who says to you that they feel that their website may have problems. You pull this website up, and you feel that there's a lot of things that can be done here to help improve that customer's performance, then when you say it's going to cost you whatever figure you want to charge. What you want to do is potentially pull that PDF to show them there are actually problems.

What we are going to do next is get Dawn Anderson on here. What Dawn's going to do is talk us through some of the stuff that Dawn feels are the more essential changes to make, or are the most common mistakes that you can start to implement doing technical because Dawn's doing technical all the time.

Dawn Anderson: Thanks for that Craig. That was really very useful. It's so wonderful that there are all these different things that are flagged up in the Site Audit Tool.

If you're working on a small WordPress site and it's got 10 pages, and it says meta descriptions are all duplicate, well that's easy to fix, but the problem is it's not easy to fix if you're dealing with a million page, dynamically driven site working off templates, because the template is going to need some programmatic implementation. That means dealing with the devs. Quite frankly, if you turned round to them and said, "Oh, we need you to do all these unique meta descriptions," they'd probably laugh at you.

So it's that kind of things that actually, whilst the tools are all great, they are just a great starting point that takes the heavy lifting out, you still need to use a lot of individual insight.

So I ran an example on a project here. For now, let me talk you through some of the processes that I use whilst I'm waiting for that to just do what it's supposed to do.

Okay, so... the things that really matter to me are obviously canonicals. If I see that a lot of pages are coming back with multiple canonicals implemented, there's only going to be one of those taken notice of, I see that quite a bit. Often that one of them is added by WordPress for instance. That is a programmatic issue that you need to fix.

I also see quite a lot of canonicals where I have to dig in and have a look at it. I see, for instance, that people have tried to canonicalize a series of page one, two, three, four, five, six or whatever, a paginated series back to the first page in the series. So I see that quite a lot. These are the areas where I've found that if you fix them it can be quite impactful pretty much immediately, or very soon if you can get a good crawl after that.

Those are the kind of things that this crawl tool, the Site Audit tool picks up on. So one actually that comes up, that actually comes up as only a notice, is one that says, "Only one internal link."

Now to me, that's not a big deal if you've got a small website. If you've got a massive website and you have a lot of pages that maybe you could be achieving long-tail traffic at scale, then that is where I would look into it much further. Also, I see people push a page out with a new post, and obviously it starts to get traffic. Then eventually it starts to get buried in the blog as new content gets added.

So when I look at, "Oh, this page has only got one internal link," and I see that in the warning section or the notices section, I am quite drawn to that because I know that even if you don't fix a lot of the errors, sometimes by focusing in and having a look at the pages which are not getting the contextual internal links in a natural way, these are areas where you can actually have really good wins.

Sometimes it only takes a little bit of good internal linking for you to leapfrog over some other sites. Particularly on some of the more longer tail queries, which often you'll be starting from if you're a very new website.

The Importance of Fixing Mixed Content Issues

One that actually is a top issue, and I totally agree with SEMrush on this, those who've got issues with mixed content. Mixed content is, for me, one of the biggest wins you can possibly have. Obviously, you've got the not secure issue.

Anybody who's not clear on what mixed content is, it's where the page is returning the HTTP (not secure), but it's trying to be HTTPS. So there is a security certificate in place, but for instance, you're returning content on the page that's included with it, such as JavaScript or images, that are not secure.  

So you're embedding or calling an HTTP image or an HTTP ad or an HTTP JavaScript file, some things included anyway, that actually is not on HTTPS. So to be truly HTTPS, you have to have all of the assets that are returned with the URL as HTTPS, otherwise, it's considered mixed content.

So that, actually something that I would consider a top issue. It always makes a difference when you fix that, for me, as far as I'm concerned, because you get rid of A, issues with the canonical being selected at HTTPS, and then the HTTP version popping up in search. You're consolidating all of your signals, and obviously, you are not getting any not secure errors and you're just focusing all the signals on the one pure and correct URL.

Making Sure You Link Internally to HTTPS Pages

I often see as well internal links to HTTP pages versus HTTPS. Again, that for me is a massive area where big wins are to be had there. Often I see that these issues are in XML sitemaps. People switch to an HTTPS protocol but they forget to update their XML sitemap.

All the links on the XML sitemap are still on HTTP, or they forget to update all of the internal links, or they forget to update all the images or something. Often these are things that you could potentially fix with some plugins if it's WordPress. Bear in mind though, you should always do a crawl before and after because a lot of the plugins don't catch everything when it comes to updating all the assets from HTTP to HTTPS on all the internal links.

So as with everything, all the tools are great, a lot of the tools are great, but they're not all perfect. So humanize, multiple crawls, double checking everything, use a checklist for things like HTTPS, switches, protocol switches and so forth. I use Trello boards as well.

The Agile Approach to Technical SEO

I have an example website audit output, which I'll actually add a link at some point. But basically, I use an agile approach because obviously, not everything will necessarily have the same level of priority as SEMRush puts on these things by the tool, because always it depends.

So what I do is I'll take the list from the errors and push it to Trello. I'll build three lists on Trello beforehand that says errors, warnings, and notices, and I can automatically just connect SEMrush and literally push the list, all of the items across to the Trello board.

Then after that, I go through with the client or the client's developers. We organize something like a discussion where we go through the list, look at it, see which are the most important, which are the easy jobs to implement.

Often it's things like the HTTPS. Sometimes it's speed actually. One of the things that do flag up on SEMrush, and obviously rightly so, has to do with speed. Speed is a massive issue. But if I then run the client's website through PageSpeed Insights and it's coming up at 80%, and the rest of the competitors are like 60-odd, then I'll say, "Well, this is a reasonably good website."

Also, it depends. I look at individual pages themselves. What's the impact on the user? What's the impact on revenue? Where is the priority? I go through the list and we look at what can we fix, what do we have the resources to fix, where is the priority, and we put labels on them from Trello. Obviously with different colors on, so color coding is a really great thing. Then we schedule sprints to get these things done. I'm in contact with developers throughout that time. We utilize things like Slack as well to keep in touch.

After that, what we do is we then run another crawl. I would say audits are not something that you should ever set and run once. I don't. Periodically, you have a scheduler in the Site Audit tool, and I set it to run once a month automatically so we can check progress. One thing that I do often see happens is that because there are a lot of dependencies, particularly on bigger websites, you fix one thing and you spring a leak somewhere else.

Also, we're all humans. So sometimes developers can do a rollback by accident to the previous code. Also, things like redirections, broken links. I'm not saying fix every broken link. If you don't sell something anymore, if something is not available, if you haven't got the content there anymore, you should return either a 404 or 410 or 301 redirect to an appropriate place. The point is it has to be an appropriate place.

Yeah, so I mean there's loads and loads of things that you can do with the tool. I found that the most success has been around things like internal linking, fixing mixed content, checking crawlability. I double check it all with things like Google Search Console, which is amongst my favorite technical SEO tools. I never use just one tool; I use a range of them.

There's nothing more embarrassing than going to a client with something that's a false positive, particularly if they're a new client or if you're doing a proposal and you say, "Hey. Such and such a thing is broken," and they go, "No it's not." How embarrassing is that?

Also, if you go to developers with things like that as well, you'll just show yourself up. You'll lose credibility. They won't believe a word you say in the future, and if you haven't got developers on board, the technical SEO and the technical audience, then you are up against quite a challenge. Could you ask me some of the questions, Craig?

What is a Good Score for A Website Audit?

Craig: So the first question we've got is from a Nicholas. He's asking what is a realistic range of score for a site audit? Is 90 good enough, and 50 being horrible, or can we actually achieve 100%?

Dawn: I have seen people who've achieved 100%. Personally, I would not strive to achieve 100%. It's all about is the effort, is it worth it? So if you have 98%, and you have something that's actually quite a complex fix but isn't actually an issue that's having an overall impact on the business, yeah, so there's not a business case for it.

So for instance, one of the questions that I did see earlier was uncompressed JavaScript. So the tool obviously picks up on every JavaScript script that gets pulled in. Sometimes those are third party. You're not necessarily going to be able to compress the JavaScript, the third-party script that's being pulled in without contacting the third party. If you have a fast site anyway, you could have got 95%, but it's these things are keeping you back. I would say so what? I would rather go off and focus on other things.

That is why taking this agile approach is really important. It depends because actually, you can have one issue that takes your score down by 40%. It can be fixed with one dev thing, and actually, it could be that it's on the pipeline, it's going to be fixed in three months. There's nothing you can do about it between now and the three months. You have to just keep chipping away at other things.

So I think with anything, numbers are funny things. It's like page speed, and you know when things are bad. You look at analytics as well and you integrate it with SEMrush, and you see, "Oh, this is a really good speed for important pages," yeah? So it's about identifying does it matter, is it a high priority. I mean, it'd be amazing to be perfect in all things, but unfortunately, that's not necessarily going to be achievable all of the time.

If you see that there are some pages that are being picked up on, for instance, that are slower. You go to analytics. You see that they're your checkout, your shopping cart, your payment page, your major priority categories, and sub-categories. Then you're putting them in the must-have bin in the Trello board.

So, it's about building this list of priorities, and then moving things up. The agile approach says we work with the most important things first, and then we keep working our way down the list based on the time that we have. If we don't have the time, we drop something off that's not important.

Craig: It depends, but tackle things in order of priority.

The next question we have is from Alyssa. How deep do you go with a site audit when speaking to a prospect? How much detail do you give prior to signing up that client?

Site Audits Before Signing Up Clients

Dawn: Okay, so again, it depends. If it's a massive website then I would take a slice of a key area and I would say, "Look, this is the top-level micro audit”, to identify a few of the issues.

If it's a smaller site, I'll just go through the whole website with one crawl. Have a look, do some insight, but obviously I never just use a site audit tool. I never use just one tool, I use a lot of tools. So I'll take various bits from top level on each, and synthesize it into where I think some of the key issues are, put it into a proposal, and then arrange a call just to talk that through, some of the key areas I've spotted.

Obviously, it's a proposal, so I'm not going to give them a full free audit. I'm going to do a top level and look at some of the low hanging fruit. Then if it's a bigger site, then I will just take a small slice, it's going to be a couple of thousand pages. Some sites are so big they don't even know how big their internal crawl spaces are, so it depends.

Yeah, I do see one of the questions on the screen from Andy Simpson, which is what are the main areas in the audit tool you go to when you want a quick scan of the site?

To be honest, I just go to the dashboards. Then I look at the Mega Menu on the issues and really just have a quick look. I mean, you have to dig in, but I look for any show stoppers initially.

So anyway, I would say a dashboard. I just have the emails so that I get the automatic email once a month that tells me that the audit's run, the things that are flagged, and I get the email that says, "Hey, you did a better job. Hey, you did a great job. Hey, you've got issues." I've set up some alerts so that if certain things happen, it triggers and I will get the email. So I find those really useful as well.

Should You Worry About Duplicate Meta Descriptions?

Craig: Right, we've still got times for questions guys. Perry is asking … if crawlability and site performance are in the mid-90 percentages, HTTPS is 99% and internal linking is, say, at 95%, should I worry about things like duplicate meta descriptions?

Dawn: No. Well, it depends. Sorry. Because if all of your meta descriptions are exactly the same word for word, think about your users. Think about the fact that Google often does re-write meta descriptions.

So I wouldn't obsess on it. Is there an area where you can improve on it? Are they dynamically generated? If they're dynamically generated, you probably can't do a huge amount to make them massively unique and also get a good call to action.

So I wouldn't obsess. But I would look, and take a look at it, and think, "Well, it's not worth spending a lot of time on that." Does the meta description meet the need of the user dependent on the query?

Craig: Just for my wee bit to add to that, I agree with pretty much everything you say, Dawn. Obviously if they're all done by an e-commerce website and you can't do much about it, fair play, but if you do have the ability to change them and it can help improve your click-through rate, it is something worth considering to some degree because, in my experience, click-through rate can be the deciding difference between getting that position one and two. It's not back inks, it's not site speed.

Other Useful Technical SEO Tools

Craig: So we have a question from Carlos. What other tools do you recommend to use alongside SEMrush? I'm assuming that's for site audits.

Dawn: So I use loads of different tools. Loads, loads. I use obviously Google Search Console. I use Google Analytics. Even though you wouldn't necessarily call it a technical SEO tool, it is because, for me, it helps with the prioritization of the task when I can see impact, et cetera.

I use things like DeepCrawl. I use SISTRIX. I use URL Profiler. I use things like WebConfs Header Check to double check things like response codes. I use Web247, which has a great suite of pinging tools, IP checking, or website IP checking rather. I use things like Minify CSS to help with minification. I use Smush.it tools to compress images. I use Cloudflare sometimes. The list could go on to be fair. There are so many tools out there.

How Much Markup / Structured Data Should You Have?

Craig: One last question is from Valeria. What about markup? Most of my customers are local, and SEMrush always shows a low percentage of pages marked up. What do you recommend for markup?

Dawn: Okay, so are you talking about Schema Markups, structured data, yeah? So for instance, you could maybe start to look at utilizing some of the schema around services, products, going to Data Highlighter.

It's not a bad thing to have low markup if you have a smaller site, because you're pretty much going to have a low amount of most things because it's a small site. But I would again look at it in combination with things like Google My Business, which is a great layer for data extraction, which you can use to feed in things like products, services, posts, et cetera. You can markup organization, you can markup people, you can markup videos, you can markup reviews. I'm talking about in your web pages.

There are some WordPress plugins out there as well that you can utilize for Schema. You can use the Data Highlighter for things like events if you do an event page, which focuses on your industry's events. Or you can use Highlighter for things like organization as well on a local, within Google Search Console.

Obviously, I know I'm making structured data and schema, et cetera, a big focus for a lot of my clients this year because for me, everybody's moving more in the direction of entity-driven search. People want to find more and more things, rather than words. Words are important of course, but task-driven search is, for me, around doing things, helping people. A lot of that is linked to entities. So yeah, I hope that answers that.

Craig: Perfect, Dawn. That is a great answer.

So that is, sadly, we're out of time, but thank you, Dawn, for being on today. Thank you to the SEMrush guys who've been answering the questions in the chat, and thank you all you guys for watching. Cheers.

Dawn: Thanks all. Bye. Bye.

All levels
Discover SEMrush