FREE online SEO course: SEMrush Site AuditHow to Get Started9 lessons
Introduction to SEMrush Site Audit
Welcome to the SEMrush Academy training for technical SEOs, today we will be taking you through how to use SEMrush site audit for your technical SEO.
My name is Ross Tavendale and I'm the managing director of Type A Media; we show brands how to find the optimal path to reach more customers online.
We have used SEMrush for the last 4 years to help our clients grow their organic traffic, to the point that it now powers our entire business.
We've used the audit tool to help with a plethora of problems, from mission critical migrations to daily technical audits, from massive multinationals to SMBs.
In this series we will be going through the technical strategy used by Type A Media on our clients' sites, and how we use the site audit tool to find and fix issues on these sites.
So, without further ado, let's get into it.
Getting Started with the Tool: Site Audit Crawler Setup
The first thing you are going to learn is how to actually set up the crawler.
SEMrush makes it really easy to get started, but there are some essentials you need to know first.
Crawled Pages Limit
This is the first screen you will see. You may be asking, why do I want to limit the number of pages?
Well, if you have an enterprise site, it's actually a better idea to break the site into sections and crawl a lower number of pages.
We do this because it still brings up all the template level issues, without needing to wait all day to crawl a million pages.
In SEO, it's all about cost/benefit analysis.
You can specify if you want to crawl just a subdomain (like blog.domain.com or a subfolder).
You can then choose what you want the bot to crawl from the site to the sitemaps.
Next we get a bit more technical. Most sites are fine crawling as SEMrush bot, however if you are struggling to crawl due to CMS or technology restrictions, Googlebot is a good option.
If you are on a hosted solution like Shopify or Squarespace, or your SysAdmin has really sensitive controls on your server, choose 1 URL per 2 seconds.
Allow and Disallow Certain URLs
This bit is where we can really define our crawl.
If there is a particular problem area on the site you want to investigate, you can define the exact paths you want to include and exclude.
If the site has lots of parameters caused by a search box or pagination, we want to exclude them from the crawl.
The best way to check for them is to click this link to the search console to retrieve the parameters Google has already recognised.
Next up is website restrictions. If you are crawling a staging server as you are migrating a site, typically, it's behind a login. You can add these details by clicking “crawling with your credentials”. If your robots file blocks 3rd party crawlers you can choose to ignore it.
Next up is the schedule. If you are going to be fixing things every week, I'd run it on a regular basis so you can see improvements over time.
As a safety measure, I like to crawl 3 times a week - so if a rankings fluctuation happens I can easily rule out technical SEO as the cause.
Technical SEO Audit Strategy
When you think about your website, think of it like your own musculoskeletal body. All of the parts are connected to each other and affect each other in some way.
So just like when the doctor hits your knee with a hammer and your foot shoots out, similarly, when we change one aspect of the site, many pages will be affected.
Therefore, one of the most important factors in technical SEO is setting priorities. To do this we use the Eisenhower Matrix, which defines:
Important and Urgent
Things that need to be fixed right away
Important and Not urgent
Things we need to make a business case for and fix over time
Not Important but Urgent
Something to be briefed to a secondary team
Not Important and Not Urgent
Also known as a “nice to have” - something that you will write down on a piece of paper and probably never do.
Within this decision-making matrix we are going to approach key aspects on technical SEO theme by theme.
- Hygiene. Basic technical checks that your site needs to fundamentally work with search engines.
- Organization. How to best structure the content of your site for maximum visibility and user experience.
- Presentation. Elements of your site that affect CTR and the way your search results look.
- Page Power. How your pages interact and pass page rank to one another.
- Code and Security. Organising the technology on your site for a safe user experience.
- Internationalization. How your site is perceived in different countries.
- Performance. How your site works across different devices and internet connections.
- Reporting. How to report on each of the issues and build a project management workflow.
How to Approach Issues, Warnings and Notices
The first screen you see in the Site Audit tool is the overview.
This presents all the top-level issues, warnings and notices as well as thematic reports about the specific issues around crawlability, security and internal linking.
The next tab is the Issues tab. This is a full list of every issue that we've been able to find. You can click into each of them to get a more detailed report.
The next tab is the crawled pages. You can view this as a list or a tree structure.
The tree structure is particularly useful to get an overview of the content silos on your site and to see how you are organizing your information for the crawlers.
The next tab is statistics.
If you are an agency pitching for new business or an SEO working in-house that needs stakeholder buy in, the graph view is particularly useful for presentations.
The next tab is the compare crawls tab. This is useful for looking at progress over time and can be used to diagnose ranking fluctuations.
The progress tab is great for reporting. It lets you look at total issues over time and even drill down into specific issues.
We particularly like the Notes section as it lets us mark up our progress so we can provide context on what's going on from a technical POV.
Now, when you first run your crawl and look at the dashboard and see all those Issues, Warnings and Notices, you are going to feel a couple of things.
Your first feeling may be staving off a heart attack as you see thousands of issues light up your screen.
The second impulse is going to pick up the phone and scream at the developer.
That made you a website with more holes than a piece of Swiss cheese. However, it's important to remember 2 things.
A developer's job is to deliver you a website, not make it rank in search. That's the job of an SEO.
Of these thousands of problems, lots of them can be fixed at a template level. This means that you are going to be able to make one fix and knock out several hundred issues at a time.
So, when we look at Errors, Warnings and Notices, they are presented in priority order.
Errors are the highest priority fixes that we need to make on the site.
These are typically issues that are going to really hold your site back from making progress because Google is unable to access or understand what's happening on your site.
Think of an error like a burst pipe in your kitchen.
It really should be fixed as quickly as possible, and when you fix it, make sure you use a permanent solution instead of using a homemade solution with chewing gum and paper clips.
Warnings are slightly less critical than errors.
Although they will have a direct impact on your performance, they won't take you out of the race altogether.
Imagine you are running a race at the Olympics but you need to carry a lot of weight with you.
You are still going to finish the race, but it's going to be way harder to win with the issues.
Notices are more like points of interest that you need to look into and diagnose.
A notice is kind of like getting a pimple - it's probably nothing, but best to get it checked out just in case there are some underlying issues.
Introduction to SEMrush Site Audit: Lesson Summary
That's everything for lesson 1 on how to use SEMrush site audit for your technical SEO, we hope you've found it valuable.
If you have any questions, feel free to tweet them to me directly @rtavs or the professionals at the SEMrush Academy on @semrushacademy.
To concrete your learning and to increase your competency, please go to the Site Audit tool and try to analyze the data and pull the reports for yourself.
Also, once you are ready, head over to the SEMrush Academy page and take the quiz for this module so you can get SEMrush certified.
If you want to learn more about this topic, you can see a list of useful links and further reading that will help inform your knowledge on the subject.
You’ve completed the course! The course is left open for you, so you can go back and watch the videos at any time navigating through the left-hand menu. Now take the next step towards perfection and try to pass the SEMrush Site Audit Exam.
Share your achievement
You are going to start the !
Before you start, review the information below:
Certify your team request
Want to rest assured that your employees’ knowledge of the SEMrush toolset is up to the mark?
If you’re interested in making a request, please leave your email.
Become a contributor
Want to join the SEMrush Academy expert community? You will have the opportunity to reveal your own tips and tricks on SEMrush tools. If you’re interested, leave your email.