How To Find Broken Links Using The SEO Spider
Dan Sharp
Posted 2 September, 2023 by Dan Sharp in Screaming Frog SEO Spider
How To Find Broken Links Using The SEO Spider
Table of Contents
- Crawl The Website
- Click The 'Response Codes' tab & 'Client Error (4XX)' Filter To View Broken Links
- View The Source Of The Broken Links By Clicking The 'Inlinks' Tab
- Use The 'Bulk Export > Response Codes > Client Error (4XX) Inlinks' Export
- Crawling A List Of URLs For Broken Links
- Find Broken 'Jump Links'
You can use the Screaming Frog SEO Spider for free (and paid) to check for broken links (the http response ‘404 not found error’) on your website.
Below is a very quick and easy tutorial on how to use the tool as a broken link checker. First of all, you’ll need to download the SEO Spider which is free for crawling up to 500 URLs. You can download via the green button in the right hand side bar.
You can crawl more than 500 URLs with the paid version. The next steps to find broken links within your website can be viewed in our video, and tutorial below.
1) Crawl The Website
Open up the SEO Spider, type or copy in the website you wish to crawl in the ‘Enter URL to spider’ box and hit ‘Start’.

2) Click the ‘Response Codes’ Tab & ‘Client Error (4XX)’ Filter to View Broken Links
You can wait until the crawl finishes and reaches 100%, or you can just view 404 broken links while crawling by navigating to the ‘Response Codes’ tab and using the filter for ‘Client Error 4XX’.
There are two ways to do this, you can simply click on the ‘tab’ at the top and use the drop down filter –

Alternatively you can use the right-hand window crawl overview pane and just click directly on ‘Client Error (4xx)’ tree view under the ‘Response Codes’ folder. They both show the same results, regardless of which way you navigate.

This crawl overview pane updates while crawling, so you can see there number of client error 4XX links you have at a glance. In the instance above, there are 11 client errors which is 0.12% of the links discovered in the crawl.
As a lazy alternative, the ‘Issues‘ tab will also alert you to any issues, warnings or opportunities found in a crawl. If any broken links are discovered, they will be reported here.

In this case, the broken links discovered are to external pages. So they are classified as a low priority warning. Internal broken links are classified as a high priority issue in comparison, but there are none found here.
3) View the Source Of the Broken Links by Clicking the ‘Inlinks’ Tab
Obviously you’ll want to know the source of the broken links discovered (which URLs on the website link to these broken links), so they can be fixed. To do this, click on a URL in the top window pane and then click on the ‘Inlinks’ tab at the bottom to populate the lower window pane.

You can click on the above to view a larger image. As you can see in this example, there is a broken link to the Moz website (https://moz.com/help/guides/moz-api/mozscape/overview/free-vs-paid-access), which is linked to from this page – https://www.screamingfrog.co.uk/seo-spider/user-guide/configuration/.
Here’s a closer view of the lower window pane which details the ‘inlinks’ data –

‘From’ is the source where the 404 broken link can be found, while ‘To’ is the broken link. You can also see the anchor text, alt text (if it’s an image which is hyperlinked) and whether the link is followed (true) or nofollow (false).
It looks like the only broken links on our website are external links (sites we link out to), but obviously the SEO Spider will discover any internal broken links if you have any.
4) Use the ‘Bulk Export > Response Codes > Client Error (4XX) Inlinks’ Export
If you’d rather view the data in a spreadsheet you can export both the ‘source’ URLs and ‘broken links’ by using the ‘Bulk Export’, ‘Response Codes’ and ‘Client Error (4XX) Inlinks’ option in the top level menu.

This should cover the majority of cases for finding broken links on a website.
However, the ‘source’ URL is the very last page to link to the 404 error page. So there might be times that the ‘source’ is a redirect (and possibly in a chain of redirects). You can see if the ‘source’ is a redirect as the ‘Type’ column will say ‘HTTP Redirect’ for example.
To quickly find the original source page of these errors, we recommend using the ‘All Redirects‘ export under ‘Reports > Redirects > All Redirects’. Open up the report and filter the ‘final status code’ column to ‘404’. The ‘Source’ is the original source page, the ‘address’ is the last source, and the ‘final address’ is the 404 URL.
There’s a number of ways you can export data from the SEO Spider, so please read our user guide on exporting.
Crawling a List of URLs For Broken Links
Finally, if you have a list of URLs you’d like to check for broken links instead of crawling a website, then you can upload them in list mode.
To switch to ‘list’ mode, click ‘mode > list’ in the top level navigation and you’ll then be able to choose to paste in the URLs or upload via a file.

Find Broken ‘Jump Links’
Jump links are a useful way to link users to a specific part of a webpage using named anchors on a link, also referred to as ‘bookmarks’, ‘named anchors’ and ‘skip links’.
These broken bookmarks can’t be found in the same way as above, as they don’t respond with a 404 status code, and they often go unnoticed.
You’ll need to enable ‘Crawl Fragment Identifiers’ located in ‘Config > Spider > Advanced’, crawl the website and then view them under the ‘URL’ tab and ‘Broken Bookmark’ filter.

Read our tutorial on finding broken jump links for a full walk-through.
Hopefully the above guide helps illustrate how to use the SEO Spider tool to check for broken links efficiently.
Please also read our Screaming Frog SEO spider FAQs and full user guide for more information.
Good tutorial. Screaming frog has become one of the most widely used SEO tool in my arsenal. Hopefully, we will see similar updates and tutorials.
4xxx error code inspection as well as 3xx has saved a tons of my time analyzing the website.
Hey Jay,
Thanks for the comment, good to hear you found the tutorial useful!
Thanks,
Dan
Thanks Dan, it is an awesome tutorial. I have tried many online sites to find the broken links. But yet this is effective!
Great feature, thanks! I was wondering if it’s possible to find broken internal links for pages that have already been 301 redirected?
Hi Nisha,
Yup, any broken links will just appear under the 4XX filter. So any links pointing to the old URL, will still appear here!
Cheers.
Dan
Soooo Awesome —- Love it!!!!
Good tutorial! And good tool! I found out that I had some broken links. Fortunately, this is fairly easy to fix and the damage was limited!
Oh yes, it is very helpful. Thanks for all !
Amazing tutorial, helpful features you got with your great tool. Much appreciated!
I’m very happy with Screaming Frog but can you help me with the following question?
Every month we have an internal 4xx check for our website. But every month the same (working) links pop up. How come that working (and safe) links appear every time? Around 60 links this time.
Thanks for this enlightening tutorial! Enough input to get started with my website. Incidentally, Screamingfrog is indispensable for me!
great tuto, everyday i discover a new feature of your amazing tool for seo puropses, thanks!
I agree, still very useful and informative after all these yaers.
Great post Dan, i love the screaming frog tool for finding broken links and doing keyword research on seasonal pages, really speeds up the process thanks
Thanks for the great information. I am able to find all my broken links on my website and improve my SEO score.
I tried it out – it is very helpful. Thank you for the help
Matthias
Yes I tried it and it’s good! Lots of info.
Brilliant tool. Used this sucker to find the golden links and 404’s. Cheers Dan
Thanks very helpful!
Is there a penalty from google for broken links?
Hi Owen,
Not a penalty as such, but it is a signal of site quality that Google use algorithmically in extreme cases in particular.
Obviously if a high proportion of your sites links are broken, it would be a poor user experience and it would make sense for Google to reduce your organic visibility.
Thanks,
Dan
Great feature, thanks! I was wondering if it’s possible to find broken internal links for pages that have already been 301 redirected?
Hi Nancy,
If the pages have been 301’d, you’ll need to look under the ‘Redirection 3XX’ filter in the ‘response codes’ tab instead.
Obviously you can view ‘in links’ of these in the same way and bulk export these via the ‘bulk export’ and ‘Redirection 3XX In Links’ option, too.
Cheers.
Dan
Iam trying to find 404 page using SEO Spider for a website which is not yet launch i.e website url like 78.23.23 .
Can anyone please help to run SEO Spider for this condition
Hi Lakshmi,
You can just input the IP address and the SEO Spider will crawl it.
Cheers.
Dan
This is a useful post for finding broken links within the website, what about links pointing outwards that are broken? I can use a free web service but wondered if this was possible within screaming frog.
Hi Eddy,
Yes, finding broken external links will work in the same way as shown above.
Actually the example I used is an external link to the BrightonSEO website, from our screamingfrog.co.uk website.
Cheers.
Dan
Hey Dan,
love the tool but I can’t figure out how to identify pages that have external 404 errors (to other sites) from a list of URL’s that I import into Screaming Frog. (The “Outlinks” Tab does not give me 404 errors)
Hey Christian,
I recommend reading this guide – https://www.screamingfrog.co.uk/broken-link-building/
Cheers.
Dan
Hi Dan,
Could you give me your email address? I need your help with the response codes. Thanks!
Hi Amy,
You can email us on support@screamingfrog.co.uk
Cheers
Dan
Hello, i was wondering, if after getting the broken links lists it is possible to export them altogether with the inLinks info. In that way you have in one report, the broken link and where in my page it is located. Thanks
Hi Dolores,
Please read section 4 of the above :-)
Thanks,
Dan
Hello – Will this find old broken pdfs in the trial mode? I ran it but cannot find pdfs that I know are broken so was thinking it turned that off in the trial mode. Thanks! Looks cool
Hi Matty,
Yes, the SEO Spider will discover links to PDF’s – https://www.screamingfrog.co.uk/seo-spider/faq/#does_the_seo_spider_crawl_pdfs
They might not be found if, they are not linked to internally, if they are blocked by robots.txt, or if you’re using the trial version with 500 URL crawl limit and you reach that limit etc (More over here – https://www.screamingfrog.co.uk/seo-spider/faq/#14).
Cheers.
Dan
Can it find broken images?
Hey Chris,
Yes, it’s the same process as above. The URL could be a broken link for HTML, an image, JS, CSS etc.
Cheers.
Dan
Hi,
I am sorry i need more details about this one too. When we run a report in crawl mode, go to response codes. Does it also scan the img src tags?
I wonder if it is possible to detect the broken images inside img src.
thank you.
Hi Nicolas,
Yes it does, it’s not just HTML anchors, but images, JS, CSS etc (anything we crawl which returns a 4XX response).
Cheers.
Dan
why screaming frog spider SEO shows connection timed out message in my system
Hi Surya,
More here – https://www.screamingfrog.co.uk/seo-spider/faq/#39
Cheers.
Dan
Hi Dan,
Just read this great tutorial after downloading the excellent Screaming Frog SEO Spider. A great SEO tool which I will now be using as part of my SEO Toolkit – thank you!
Great tool! I am using a redirect plugin to send all my 404’s to my home page but I think it’s slacking sometimes. I noticed that some other sites have taken over my 404’s somehow (I see them in the Google search with my title but their link)
Great info! Sometimes it’s just too obvious.
Thanks fot this tutorial!
Simple and effective! Cheers :)
Great stuff! Thanks for the wonderful tutorial. :-)
Hey Dan,
I have got a problem with my screaming frog software. There are so many link in my website but when I put my site and press the start button, it only shows the web link in the below table not the complete website. Please advice how I can fix this problem?
Hi Waleed,
We have an FAQ which covers problems crawling a website – https://www.screamingfrog.co.uk/seo-spider/faq/#14
If you’re not sure, then you can always pop through to support here – https://www.screamingfrog.co.uk/seo-spider/support/
Thanks,
Dan
Excellent!
Great tutorial, thanks!
Great tool. Very useful!
Great tutorial!
Thanks for easy & quick tutorial. One question I would like to ask,
Is there a way to find how many site are redirected on particular site.
Hi Rachel,
If you look under the ‘response codes’ tab and the ‘3XX Redirection’ filter, you’ll see any redirects.
Cheers.
Dan
Thanks for quick reply Dan,
I checked it but 3XX only showing same domain redirection urls. I want to know how many other websites (external) are redirected on my website.
Hi Rachel,
That tab and filter does show internal and external redirects, so if there are no external URLs there, then there are zero external redirects.
You can also view external URLs crawled (and status codes) under the ‘external’ tab (make sure under ‘configuration > spider’ you have ‘Check external links’ checked and potentially ‘Follow external nofollow’).
If you have anymore queries, then pop them through to support – https://www.screamingfrog.co.uk/seo-spider/support/
Cheers.
Dan
Thanks for this post, as I love Screaming Frog and the ability it has.
Appreciate the post. Ive been trying to find a more effective way of finding the broken links on sites. Screaming Frog accomplishes that with ease…now i just have to figure out the rest of the features
Great tool! Very useful to find broken links. Thanks for share.
Hi,
to find broken links is working for one single URL, correct?
Is there a chance to submit BULK URL list for checking for broken links?
Thanks
Dan
Hey Dan,
Good question, there is! You can upload a list of URLs in ‘list’ mode (https://www.screamingfrog.co.uk/seo-spider/user-guide/configuration/#15) and crawl them to discover broken links, too.
I’ll add this into the guide above.
Cheers.
Dan
thanks, but the limits tab can’t be changed.
Is it only possible for the paid version?
Thanks
Dan
Hi Dan,
In ‘list mode’ the crawl depth is set to ‘0’ as default, so only URLs in the list are crawled.
In the licenced (paid) version of the tool, you can amend this though!
Thanks,
Dan
Thanks for this. Someone introduced me to Screaming Frog last month and I’ve been using it for a 3/4 weeks now. Locating and sorting out broken links will be a useful addition to my knowledge of SF. Cheers
Hello,
Can I use this tool for my IP address .
Site migration not pointed to DNS
Wanting to correct errors I checked manually and have many errors wanting to verify.
Thanks for your reply
You can just crawl an IP address :-)
Hey,
I uploaded a list of URLs for checking broken Links. The tool found 52k URLs but in total only 1640 URLs are checked. Is there a limitation of URLs I can check?
Thanks
Hey Mimi,
There’s no limit, but we do de-dupe the URLs upon upload.
It sounds like there were lots of duplicates in the list! If you check Excel & use their remove duplicates function, it should be similar.
Cheers.
Dan
Hi Dan,
Thanks for your quick answer.
Indeed I have duplicates in the list.
By the way, that tool is awesome!
Best
Mimi
Hi, I use ScreamingFrog to find new domain for my social networks. It’s very usefull for searching expired links in .gov and .edu sites.
Best regarts,
Kordian
Great tool! Thank you for sharing.
I hope this doesn’t sound like a silly question, but I’m assuming the program ignores (doesn’t “click on” or “follow”) Google Adsense ads or any other pay-per-click ads one has displayed on their site?
Hi Katt,
No, the SEO Spider doesn’t render JavaScript and won’t click on ads.
Cheers.
Dan
Thanks!
Great stuff, thank you for the useful info.
Hi there!
This tool is very usefull when I need to find any broken/expired links in .gov and .edu websites.
Best regards,
Paolo
Thx for this awesome software and the useful tips. I am working with Screaming Frog every day and am thankful for this tool that makes my SEO life much easier:D
Hey, I have no words for this tool. I am currently handling more than 50 eCommerce websites with over 10,000 pages but this tool has made my onsite work veryyyyyyy easy.
Great work on the tool, you helped me identify lots of problems.
We can’t find external links in trial version ? only internal 301 and 302 redirects are being shown
Hey Shafiuddin,
External links will be under the ‘external’ tab if there are any.
Thanks,
Dan
Hi,
I have crawled a website to find broken external links. But the result is not complete. I have a full version.
Thanks,
Kerstin
Hi Kerstin,
Sorry, I am not sure what you mean by the result is not complete. I’d recommend popping through the details to support, so we can help! – https://www.screamingfrog.co.uk/seo-spider/support/
Cheers.
Dan
Hi Dan,
when I crawl the hyp-a.de, I get a list of all external links of the whole website. But the page http://www.hypo-a.de/sponsoring/nachwuchsfrderung.html contains a broken link to http://www.martingaber.de. This link is not discovered and not marked as broken.
Thanks,
Kerstin
Hey Kerstin,
The reason this isn’t found as a broken link, is because it’s not a broken link with a typical 4XX error response. The server returns a ‘DNS lookup failed’ response as its status.
If you look under the ‘external’ tab, you can see it, but the best way to catch these (different) errors, is also to consider looking at ‘No Response’, which can be found under the ‘Response Codes’ tab. You can see this example here –
http://i.imgur.com/X2NwmzO.jpg
Hope this helps!
Moving forward, please do send any specific problems through to support as per the link above (rather than commenting on an old blog post, which can be difficult to maintain!) :-)
Cheers.
Dan
I am having an issue using the free tool to look for dead links. I am hoping you can provide some assistance.
We will be migrating our SharePoint environment to Office 365.
So for instance, http://contoso/sitedirectory/test/default.aspx will be migrated to https://contoso.sharepoint.com/teams/test/default.aspx
I able to use the tool to crawl the source site and it works great! Unfortunately, when I try crawling the target site, it only finds the one site with a status of forbidden. I have double checked the permissions to both sites so I am not sure why I am getting only one site and how to get past this error.
Thanks,
Justin
Hey Justin,
Here’s an FAQ on the 403 forbidden message – https://www.screamingfrog.co.uk/seo-spider/faq/#19
If you’d like us to take a look, then send the details (including the site address) to support – https://www.screamingfrog.co.uk/seo-spider/support/
Cheers
Dan
Okay thanks Dan…I did send the details using that link yesterday. Just waiting to hear back now.
Hey Screamingfrog Team,
I have the same problem. Is it possible to contact with you guys to take a look on mine also.
Thanks.
Hey Angel,
No worries, support over here – https://www.screamingfrog.co.uk/seo-spider/support/
Cheers.
Dan
Great tool, i’ve been using it for most of my sites
Very useful tool for searching for broken links and expired domains, in high quality sites. I love it.
Hello Screamingfrog team,
I need to check my website for broken links after authentication, i.e. links are available only after logging into the system.
Does Screaming frog support this? Thanks!
Hi Anahit,
Yes, it does support basic and digest authentication, more over here – https://www.screamingfrog.co.uk/seo-spider/faq/#35.
Don’t forget the paragraph about robots.txt as well!
Thanks,
Dan
hi there
its a very good blue print – thank you fot that!
where can i find another explanation working with the frog?
many thanks
Very useful tool for searching for broken links and expired domains, in high quality sites. I love it.
Screaming Frog is the best tool when it comes to Technical SEO 100% recommended!!!
Will this software find broken links from other websites that point to mine? Like if a person posts a link to a product in their blog but that product no longer exists? I’ve moved to a new website and want to link up old urls to the new ones. There are lots of pins on Pinterest going to the old product urls.
Hi Rebecca,
If you don’t link to pages that error internally on your website, it won’t find any broken links.
So if they are old URLs, which have external links pointing to them, then I’d recommend using –
1) Google Search Console – It’s free. Google crawls the whole web and will show you any errors from external links.
2) A tool like Majestic (my fav) or Open Site Explorer (which are backlink analysis tools). You can view a ‘top pages’ report and they show status codes, too.
Cheers.
Dan
Thank you Dan!
Hi Dan,
helpful explanation. thanks a lot
Thanks for the step by step tutorial.
404 pages are also shown on the web master tools right?
Then you can fix them by using 301 redirect for example.
So can you please tell me what are the benefits of finding them using Screaming Frog?
Thanks
Hey Shai,
No worries. Yeah, you can see 404s in Google Search Console (formally Webmaster Tools) and I highly recommend using as well.
So a good question. The benefits of using the Screaming Frog SEO Spider are –
1) It’s instant. Obviously you have to wait for Google to crawl your site (or page!) to find the broken link(s). With the Spider, you can crawl anytime instantly. Whether that’s pre launch, a site migration on a staging server, or when you’ve just made a lot of changes etc. So you can fix them, before publishing and before any potential problems. Conversely, if you or a developer has fixed some broken links, you also have to wait for Google to re-crawl them for them to disappear in Search Console. So, how do you know they are all fixed and none have been missed? In the Spider, you can verify it, by re-crawling and double checking they have been fixed straight away.
2) Google doesn’t show you the ‘inlinks’ (the pages which link to the 404s) very well. It’s really clunky, you have to click each individual broken link to view in a little window, and you can’t bulk export errors alongside the source ‘inlinks’ like the Spider, to just simply fire over to a developer. They also have a limit of 1k errors, and the SEO Spider will show ALL found (some large sites have thousands obviously). We have given John Mueller and their team feedback on these points, as it’s frustrating.
3) Google doesn’t show you external broken links. I.e, if you link out to external websites and those links are broken, you wouldn’t know in Search Console.
4) It’s easier to prioritise which broken links to fix in the Spider. In the ‘lite’ free version, you can analyse the number of ‘inlinks’ a page has, or the ‘level’ (how deep a page is) which are both useful measures to consider of how important a page is, particularly on large sites. If you have the premium version, you can match against Google Analytics or Search Analytics (from Search Console) data, so you can fix broken links which historically have the most traffic, conversion, rankings etc.
5) The SEO Spider also crawls directives like canonicals or rel next/prev and will tell you if URLs there are broken. Google don’t show you those. The configuration options in the premium version, also allow you to crawl links with ‘nofollow’ etc, we again, you’d have to remove to get Google to crawl.
6) To use GSC, the site will need to be verified by you, or you’ll need to get the site owner to verify and grant you access. This can be a pain if you’re an external party etc and waiting on slow clients / devs. With the SEO Spider, you can just start crawling.
I think those are the main points! Obviously the SEO Spider will also report on all response codes (like redirects, whether they are 301s or 302s, server errors etc) and plenty of other data at the same time anyway, so the tool is more than just finding broken links.
There are benefits of using Google Search Console as well, though for balance – Google crawl the web and can find broken links via other websites, which link to old pages (or just incorrectly). For big audits, we often combine the two.
Cheers.
Dan
Great tutorial and tool indeed. Thanks ;)
I have just started using Screeming Frog and I absolutely love it! I love that it’s so fast and easy to use. By far one of the key tools I have now. Thank you for developing it and making it available for others to use.
This tutorial was really well laid out and easy to follow. Thanks!
Someone recommended Xenu for checking for broken links, then I remembered this post. I use Screaming Frog almost daily now when quoting clients and diagnosing on-page SEO issues. Thanks again for making such a great tool.
Thanks you very much! I as an IT student who finished the study work on this topic. Tutorial was very helpful. :)
Hi,
I am wanting to use this tool as a way to check if the external links in my content pieces are 200 response, 404, etc… I understand that this can be done using the spider mode but when I try using the list mode, it gives confirmation of the page response (as described in this article) but it doesn’t check the external links on the page. Make sense?
So is there a way I can upload a list of pages and have the tool crawl the external links on that page too?
Hi Jonathan,
List mode will only crawl URLs in your list, hence not the external URLs as standard.
However, in the ‘licenced’ version of the tool, you can change the configuration for ‘limit search depth’ from 0, to 1.
This would then crawl the external links.
Cheers.
Dan
Hi Dan,
Thanks! However, this is still only in the spider mode? I do have a licensed version..
I have tried in List mode, with the limit to 1 but still not crawling the external pages.
Hi Jonathan,
It will work in list mode, but probably best if you just email through the details to support – https://www.screamingfrog.co.uk/seo-spider/support/
We can then take a proper look at the example.
Cheers.
Dan
Love screaming frog, it’s proving very, very useful :)
i just love this software! works with it everyday, i think its one of the best for seo and webmasters!
keep up the good work!
Great tutorial for how to find the broken links. I think this the best seo tool. Keep sharing, Thanks!
This software is great, I use it all the time.
This is what I would like to appreciate. I downloaded your free Screaming Frog broken link checker tool.
Thank you so much!
Very useful tool for searching for broken links and expired domains, in high quality sites.
Great and very useful tool, thanks for this.
Absolutely love screaming frog. A must tool to have in your SEO kit, and if you haven’t got, then i recommend you get it.
Is it possible to show the text that a broken link is linked to? So far it seems as if the Broken Link Report produces a list of broken links, but does not show the text that the broken links were found in.
If this is possible, can you explain how I can do this? Thank you for any help that you can provide.
Hi Charles,
It shows this as default – The link text is the ‘anchor text’ column, both in the lower window pane, and in the bulk export.
Cheers.
Dan
Thanks Dan!
Hi,
Absolutely new to Screaming Frog. I tried to use it to crawl my business website. Unfortunately, the status shows as ‘DNS lookup failed’. What’s the issue and how can I get started? Thanks.
Hi Vikram,
That’s generally due to an incorrect web address, so double check that!
If not, support is over here and I recommend you pop through the details to us directly – https://www.screamingfrog.co.uk/seo-spider/support/
Thanks,
Dan
Hi
I am a newbie to Screaming Frog but overall first impressions is that it is a valuable SEO tool. My 404 errors do not seem to be so straight forward. The pages linking to the page not found either do not exist or, after searching high and low I simply can not find the suspect link to the page not found. Do you have any articles on how to use screaming frog for ideas of further analysis?
Hi Claire,
Right click on the URL with the broken link, then choose ‘export > crawl path report’.
This will show you the exact path the URL was discovered :-)
Cheers.
Dan
Can ScreamingFrog find # links? By that I mean href=”#”
Hi Bob,
No. Similar to Google, we strip anything after the hash value.
We will be making this optionable in the configuration though, so you can find those anchored links.
Cheers.
Dan
Any update on this?
Hey Johan,
Yeah, you can do this now. Switch to JavaScript rendering mode (Config > Spider > Rendering), and the SEO Spider will crawl URLs with fragments and treat them as seperate URLs.
(You can also switch off this behaviour by using URL Rewriting).
Cheers.
Dan
Seems great tool. I am going to use it.
Greetings
I have only one question – is it also available in the free version
Thank you so much
Leonid Ostrovsky
We are using this software for all of our broken links and total number of pages – this is amazing software, with the best results. Thanks to the Screamingfrog team.
Brilliant broken link tool & information – thank you for providing!
Love screaming frog. One of my most used tools :)
Amazing tutorial, thanks so much guys, only just started using screaming frog for my new business so looking to suck up as much knowledge as I can to help me.
Good tutorial. Will try the software. Thanks for sharing.
Best Software ! I got to hear it from a friend who used it for finding broken links and now im Using it ! Easy to use !
I tried many tools to find broken links for my projects. But this one is really great. Interface and function of this software is itself great and helps to fix 404 error very efficienlty.. Thanks you!
Thanks sir. great tool
This is one of my favourite tools. While the free version is good, the paid one is superb.
Amazing what you can get out of this tool. A must when working with online marketing.
Great tutorial. Its the first time I have seen this tool – Thanks for sharing!
Without a doubt the most used tool that I use on a day to day basis. Keep up the good work.
Awesome article. Great information you have shared – This will help us a lot!
Sir, I downloaded the SEO Spider and and executed it, but it is not working. Why?
Hi David,
FAQ over here – https://www.screamingfrog.co.uk/seo-spider/faq/#why-wont-the-seo-spider-start
Generally just due to requiring to update Java.
Cheers.
Dan
Hi Dan,
It would be really helpful if 404s that appear in the crawl list because of an incorrect canonical actually show on the Inlinks tab, or… anywhere.
Instead I’m being presented with this message: “No Inlinks discovered. Possible reasons: first url in a crawl, url uploaded in list mode or url discovered from canonical link element.”
This is really problematic for any site, let alone the current one I’m working on with (40k+ urls).
Is there a way to view the source of the 404ing canonical, or, is that not possible?
Cheers,
Joe
Hey Joe,
Thanks for the comment. When we updated the SEO Spider to allow crawling of canonicals, we decided not to class them as an ‘inlink’, as they are not a proper HTML anchor tag obviously. So they don’t appear in the usual ‘4xx client error inlinks’ export currently, or as an ‘inlink’ in the lower window tab. This is something we are debating internally, as it would be useful, even if the definition is not completely correct etc.
However, you can still get this data – Use the ‘Reports > Canonical Errors’ export from the top level menu.
This will show you the ‘source URL’ of any canonicals that are not a 200 response, a 3XX, 4XX, 5XX and let you know if they are ‘unlinked’ (meaning the only way we found the URL was via a canonical, it doesn’t have a proper link on the site to the URL).
Hope that helps!
Cheers.
Dan
Great tool! Not only is it good for checking broken links (as described in the article), but it’s also been very helpful with identifying many other issues with clients websites. Keep up the good work ScreamingFrog!
Hi. Can i use license version on 2 machines (PC+laptop) . Thanks in advance for answer.
Hey Elkomp,
More in our FAQ here – https://www.screamingfrog.co.uk/seo-spider/faq/#can-i-use-my-licence-on-more-than-one-device
Cheers
Dan
Just downloaded the tool and used it. Working perfectly fine. It discovered 21 broken links on my website. Really helpful tool.
Great tutorial and great software. Really useful.
I need this one! Thank you very much for this!
Just downloaded the tool and used it. Working perfectly fine. It discovered 21 broken links on my website.
Great Tutorial. Seems to be a good Software
There is no doubt that Screaming Frog is the best tool when it comes to Technical SEO. Thank you guys!
Hello – Great tutorial and great software!
Your tool keeps getting better and better well done and thanks for the tutorial to help us learn new features we really appreciate it.
When I use Google Search Console / Webmaster Tools, it reports that I have 138,000 broken urls on my website. When I did a crawl using ScreamingFrog and filtered for 404 errors, I find fewer than 2,000. What explains this discrepancy?
Hi Ron,
The core differences are discussed here in our FAQ – https://www.screamingfrog.co.uk/seo-spider/faq/#why-does-the-number-of-urls-crawled-not-match-the-number-of-results-indexed-in-google-or-errors-reported-within-google-webmaster-tools
Hope that helps!
Dan
Thanks for the post Dan.
I think scream frog is one of the best software out there for on-page analysis. Thank you. One question. How does it find broken links pointing to a website (404 errors) does scream frog use a third party software like ahrefs? Thanks
Selam peogram seo açısından çok faydalı görünüyor. En fazla kaç kullanıcı girişine izin verilmektedir? Birden fazla kullanıcı için ek ücret ödenmesi gerekiyor mu?
Hi,
Licences are individual per user – https://www.screamingfrog.co.uk/seo-spider/faq/#how-many-users-are-permitted-to-use-one-licence :-)
Cheers.
Dan
Best Software ! It would be really helpful if 404s that appear in the crawl list !
Hi,
One question:
In my case I extract a big list of errors from Search console by Google, and this list I upload it to Screaming Frog to check its status, but if I want to know from which urls those errors come from, I can´t.
I only can know from which url came from an error if I crawl the entire site, but for websites with a huge number of urls it is too complicated….
Can someone give me feedback about it?
Thank you!
Alberto
Hi Alberto,
To know the source of the errors, you’ll need to crawl the whole site, as obviously the SEO Spider won’t have that data from just crawling a list of URLs.
So, the best option might be to crawl the list of URLs from Search Console, and match those that error, with a complete crawl and the source.
There’s no easy way to do it, other than matching up the data I am afraid.
Cheers.
Dan
So far it seems as if the Broken Link Report produces a list of broken links, but does not show the text that the broken links were found in.
Hi Mallo,
Have another read of number 4 in the guide ;-)
Cheers
Dan
Hi Dan,
Thanks for the helpful article.
Awesome guide on finding broken links, I found this super helpful.
Love the tool! Thank you!
Haiden
Great tutorial. Its the first time I have seen this tool – Thanks for sharing!!
Great tutorial – the software is more than useful and really is an asset for any online marketeer.
Great tutorial, thanks, it help me a lot as I had a bunch of broken links on my domain !
Hey It is possible to find internal broken link also, from Screamingfrog?
Hi Akshay,
Yes, using the same method as described above. The link found just happened to be an external 404, it could have been internal!
Cheers.
Dan
Thank you for the all updates over the years.
Screaming Frog SEO spider has been my go to tool for on-page seo for years.
I am finding this software really useful for marketing my articles. Good work!
At first I thought – they’re not going to have a Mac / OS X version of this, then I was proven wrong! Thanks so much for this.
Hi, this is a nice website tool which corrects all broken links and provides the on page issues
I recently started using this software and find it a necessary tool now. I highly recommend it. Good work!
Hi guys, I use Screaming Frog Tool to find attactive expired domains, which I use to build my PBNs. We can also use found domains to get some strong BLs by 301 redirection. SF seems very usefull for searching expired links in edu or gov websites. Regards, Pawel
Thanks guys for your work – year by year. A “must Have” tool for me. Extremely useful, free for small companies, low entry fee for bigger sites. One of the best crawler on market.
Great post. Thank you very much for detailed guidance. It is very helpful.
Thank you for this great tip and you helped me a lot to use seo spider tool
Really good broken link checker tool thank you
We use this tool a lot for onpage seo analysis. Great tool which simplifies otherwise complex processes.
Fantastic tool, we use it mostly to find broken links on our client’s websites.
Thanks
The SEO Spider has saved hours of work for us.
Thank you for this tutorial.
I had used to check the broken links manually before one of my friend recommend this tool. It’s an effective way to save time.
Hi Dan, I can’t say enough of how much this tool has helped facilitate my SEO projects. Thank you for this informative tutorial!
An absolutely superb tool, and many thanks to Dan and team for producing. A must for effective SEO.
Very nice software to help check for SEO errors.
I got this from backlinko, great tool for broken links, seemed can save me a lot of time, can’t wait to give it a try!
In the past I use one chrome plugin to do this, it costed me a lot of time, I should find it earlier!
I’m a web developer and was looking for a good web crawler to check fro broken links. First time using Screaming Frog and it’s awesome.
Thanks,
Hello, I am a digital marketer who is new to your tool, great job by the way. I am currently trying to make my way through your user guide, However, one question I cannot find the answer to is why does the tool report 404 errors for some URL’s with ” at the end? I have checked the corresponding pages on the backend with WordPress and they don’t seem to exist. Any idea why?
Hi Oisin,
Thank you! I’ve never seen the SEO Spider report these errors, so it must be related to a particular site and scenario you’re crawling.
Please do pop through an example to us via support here and we can take a look for you – https://www.screamingfrog.co.uk/seo-spider/support/
Cheers.
Dan
OK, I’m really impressed with this 404 broken link checker, this really helped us regain our search engine ranking, which dropped as a result of menu link change which rendered many pages on our site unavailable, but we were able to get all the broken link using Screamingfrog.
By the way I gotta say, this app is really fast to compare with other tools I have used in the past, too.
Can you make the option to find 404 pages from Google SERP? For example, if I use keywords on Google “site:ry.ae” I see a lot of 404 pages crawled. It will be great and amazing help for us, SEO experts to analyze crawled pages from Google directly.
Just had to say thank you for Screaming Frog, and this tutorial.
I’ve been scratching my head a little when it comes to tracking down broken links.
This tutorial was a great way to discover why so many SEOs whom I respect are always recommending Screaming Frog.
Thanks
Hi there,
May I know if your plugin can do the following?
1) Search for broken links on a website and do it with different IP addresses? Reason is that my site has links that redirect users to different links based on their their country. So I would need to check whether the various redirects when users click on the link work as well.
2) Searches hidden pages on my websites for broken links.
Thanks.
Fantastic broken link checker tool, thanks!
Great Tutorial. Thank you very much for detailed guidance. It is very helpful. I love the screaming frog tool for finding broken links and doing keyword research on seasonal pages, really speeds up the process. :)
Screaming Frog has become indispensable in my audit process, thanks!
A friend mentioned Screamingfrog and told me it’s brilliant.
After checking this out, it seems he wasn’t exagerating.
Many thanks for the superb tutorial, and I shall be using this for my work.
Very kind, James! Appreciate the comment, and good to hear the tutorial was helpful!
It’s a very helpful guide to find broken links, not only for freshers, but also experienced SEOs.
Screaming Frog has become indispensable in my audit process, thanks
Thanks! This tool is very helpful at all to find broken links. For free it is very limited yes, but that’s okay, I’m quite satisfied with this one tool.
It was very useful for SEO knowledge, thanks!
Just had to say thank you for Screaming Frog, and this tutorial. I’ve been scratching my head a little when it comes to tracking down broken links.
This tutorial was a great way to discover why so many SEOs whom I respect are always recommending Screaming Frog.
Thanks!
We use Screaming frog in our office. The program has the most useful features. Thanks for the broken link checker tool.
Screaming frog is definitely a one of a kind software in the SEO world, and what you have shown in this post clarifies this even more! Thanks for making this tool and continuing to develop it.
Screamingfrog Thanks, It’s very good knowledge about broken backlinks (404) in many different ways.
Thanks for great tutorial!
Thanks for the great tutorial. Just solved a client problem super easy with this.
Do you guys plan to add broken youtube links checker to your program? Cause now it doesn’t see them.
Hi Max,
I just crawled YouTube and it seemed to work without any issues.
You may find some links are ‘nofollow’ and hence you’ll need to the ‘crawl internal nofollow’ under ‘Config > Spider’ ticked.
You may also find some elements are JavaScript, which may require JavaScript rendering activated (Config > Spider > Rendering’).
If it’s a link, the SEO Spider will be able to find it.
Cheers.
Dan
I just wanna say THANK YOU for this wonderful guide!
(And this great tool!!!)
carry on!
I’m new with Screaming Frog, but I have just understand the power of this tool.
In detail broken link, that are hard to find.
Thank you very much and good job.
Liz J
I have tried so many tools, but your tool is really fantastic. I just want say thank you so much.
Thanks for the great support. Just solved a client problem with tool.
Never thought these many techniques can be used. really a very thorough and well organized content
Good Post!
Great content!
Best crawler and SEO tool ever, thanks for posting this.
Is there a way for screaming frog to search broken links within a page from a website? Not just a broken page within a website.
Hi Soyeon,
Yes, you can upload a single link in list mode, then change the crawl depth to ‘1’ under ‘Config > Spider> Limits’ and it will tell you all broken outlinks from the URL.
Cheers.
Dan
Thank you so much!
When in ‘Spider’ mode, looking at the directories, I see ‘https’ & ‘http’ folder with the structure of the site and all the same links.
Shouldn’t it show only ‘https’ since the website has switched from http and is now running only on https?
Hi Klaudio,
It will show whatever is linked to internally.
So if you link to HTTP URLs, it will show them. You probably want to clean those up :-)
Cheers.
Dan
I want to thanks Screamingfrog because it’s helped me a lot to determine which of my content is duplicated and which URL should be removed permanently. With the help of your software, I solved my problems related to duplicate content which was sent by a Google Adsense reviewer. Thanks again to Screaming frog.
But when I came to your software it was a little bit difficult to determine that where my problem will be solved. I spent a lot of time to understand the structure your software. But, finally, I solved out my problems here.
Great article, thank you! :)
Just gonna say thank you to Screaming Frog team for this tutorial. I’ve been scratching my head a little when it comes to tracking down broken links. This tutorial was a great way to discover why so many SEOs whom I respect are always recommending Screaming Frog. Thank you so much!
Screaming Frog is a one of the best tools to search engine optimization. Very simple and useful.
Good guide.
Thanks to you and Screamingfrog for this tutorial, very useful
Thanks for the broken link checker tool, is fantastic for big sites.
Every doubt solved.
Thanks again
Screaming Frog tool is awesome! Thanks for letting me know how to find broken links.
Find this tools very, very useful. I will definitely get the paid version as the free one only allows up to 500 pages crawl limit. Best tool to find broken links! :-)
Best crawler and SEO tool ever, thanks for posting this.
Thank you, just download the free version to find the broken links. Am happy that it worked so well for me and it is recommended.
We were finding the best solution and guess we have got it. Going to download now. Thanks
Hello
This software is translated into other languages such as Spanish.
It has a user guide if you buy a paid version
Thank you
Very useful. I will definitely get the paid version, as the free one only allows up to 500 pages crawl limit. Best tool to find.
A must-have tool for any SEO Audit.
Is this tool helpful when u are checking for broken links in an intranet portal of a software vendor website which require login credentials everytime you want to access the webpages, for example broken links for partnerworld internal portal!!
Yup. The paid version allows you to login with basic and digest authentication, and any web forms. :-)
One of the best SEO tools for an SEO audit. Thanks for the tutorial.
Hi Team, Its a great tutorial however the image is not clear even though I have tried to download and zoom it.
It will be of great help if you provide a video tutorial along with it.
Thank You
Great tool for SEO analysis and improvement, I think it’s a must-have tool for any professional.
Good work.
Awesome, There are many broken link checks available online but the screaming frog is the most powerful I ever came across.
thank you – Best crawler and seo tool ever thanks for posting this
Thanks for the tutorial. I was following that and fixed broken pages on my webshop ! great job !
Hi,
Thanks for this amazing guide of broken links checking using the seo spider tool.I downloaded it and get awesome results.
Great tutorial shared.
Thanks
I am scanning my site and have a few 4xx errors with inlinks that are not on my domain.
Does that mean it is a broken link on someone else’s site?
My site is http://www.dutchcrafters.com
and the 4xx errors include
inlinks from
http://www.familycircle.com/holiday/easter/decorations/gorgeous-easter-eggs/?page=4
to
https://www.familycircle.com/holiday/easter/decorations/gorgeous-easter-eggs/
It seems to be an 3xx since it is http to https. But it is listed under 4xx error and the domain is not mine.
Hi Mike,
Yes, they are external links that you link out to which are 404s.
I’d recommend looking at the ‘inlinks’ to the URLs above to see where they are linked to from, as per the guide.
Cheers.
Dan
Great tutorial with very detailed instruction! I have a question, is it the 404 links reported by ScreamingFrog are the same with the 404 crawl errors reported by the Google Search Console? Thank you! :)
Cheers, Martha.
I recommend having a read of this FAQ about the differences –
https://www.screamingfrog.co.uk/seo-spider/faq/#why-does-the-number-of-urls-crawled-not-match-the-number-of-results-indexed-in-google-or-errors-reported-within-google-webmaster-tools
Cheers.
Dan
Is there a way to export the broken page and the location, for example
example.com/four-0-four
Located on:
example.com/page1
example.com/page2
example.com/page3
so we can send to clients for them to be able to fix the URL’s, currently I have to go to the inlinks section and copy and paste and format in an excel which takes a lot of time.
Great tool, by the way.
Hi Ronan,
Number 4 in the guide above :-)
Cheers,
Dan
Thank you for the helpful article. It’s better to have such written content. It’s easier than watching tutorial videos in my opinion.
Graphically could still be optimized but functionality is very good and Recommended.
Hi Dan,
Just curious as to how long Screaming Frog stores a 404 error after a crawl. Reason I’m asking is that I am finding 404 links on Screaming Frog that I’m no long seeing these 404 on Google Webmaster Tools.
Thanks!
Hi Nicholas,
The SEO Spider reports the response codes returned from it’s requests at the time of the crawl.
So it’s at the moment you run the crawl and the URLs HTTP response is returned, if you keep the crawl open for a week, obviously they will be a week old etc.
Google Search Console is obviously different, it’s based upon when Google crawls the URLs over time. So this might be before the SEO Spider, or after.
More here – https://www.screamingfrog.co.uk/seo-spider/faq/#why-does-the-number-of-urls-crawled-not-match-the-number-of-results-indexed-in-google-or-errors-reported-within-google-webmaster-tools
Cheers.
Dan
I really have no idea how could i do my job without your amazing software, Thank you so much guys for the updates, ill always recommend about you!
Very useful tool for searching for broken links and expired domains, in high quality sites. I love it.
Thoughtfully designed tool to get the complete info on broken link. I used this tool for my site and got really amazing results. Now I have no worry to think about broken links about any site. I have any single tool with this perfection. Highly recommended. And the comments section too very helpful and thanks to the author for addressing each and every comment.
Cheers, Srisadhana!
Thank you for the helpful article. It’s better to have such written content. It’s easier than watching tutorial videos in my opinion.
I really love the tool, it’s just amazing how much you can do with it.
However; i recently noticed a problem, when analyzing a website. It doesn’t seem to find all the 404-errors. There are pages, where when I check it with the chrome developer tool, I clearly get a 404-error, but when I crawl that same page with Screaming Frog, it shows me a 200. What could be a reason for that?
Hi Matthias,
Thanks for the kind words, and awesome to hear!
If it’s not finding the link, then just the usual reasons (it’s not linked to in the raw HTML, blocked by robots.txt, nofollow etc) – https://www.screamingfrog.co.uk/seo-spider/faq/#why-is-the-seo-spider-not-finding-a-particular-page-or-set-of-pages
If it’s finding the URL, but not reporting it as a 404 (and sees it as a 200 response, different to a browser) then – https://www.screamingfrog.co.uk/seo-spider/faq/#why-am-i-experiencing-a-different-response-in-a-browser
Send any specific examples to support and we can always tell you why, but it’s 99.9% of the time one of the above. :-)
Cheers.
Dan
Very useful tool for searching for broken links and expired domains, in high quality sites. I love it.
Hi Dan, I have crawled my website and some of the pages having 0 in status code, but they are normally opening in browsers with 200 OK status. What could be the possible reason for it?
Hi Cryotos,
There should also be a ‘status’ alongside the status code of 0, which gives you a better indication of the issue.
It’s generally a connection refused, or timeout etc. You can read more about and how to resolve them here – https://www.screamingfrog.co.uk/http-status-codes-when-crawling/
Cheers.
Dan
Thanks a lot Dan!
Great article. I use broken link checker WordPress plugin to check broken links. Its very easy and useful.
Well explained & helped me to deal with the broken links to my site. Thanks
Hi folks, does this utility work on sites hosted on WordPress.com? If so, should a user use their custom domain address (i.e. mydomain.com) or their underlying WordPress.com subdomain (i.e. mysite.wordpress.com) address in the “enter url to spider” box? A user posted in the community forums that they tried using it on their site and SF came back with “Connection Refused”. Any help appreciated. TIA
Hi JenT,
WordPress.com servers respond with a connection refused to a request from the Screaming Frog SEO Spider user-agent.
This appears to be some kind of ‘bot’ related check the server obviously performs, to ensure only real people visit etc.
You can still crawl the websites by changing user-agent (Config > User-agent) and switching to Googlebot, or Chrome etc. But this feature is only available in the licenced version of our software.
If anyone has any connections at WordPress, it might also be worth asking them to whitelist the user-agent (as it’s obviously non-malicious) so everyone can use it for free.
Hope that helps!
Thanks,
Dan
Hi Dan, Thanks for the quick response. I’ll let the user know.
does this utility work on sites hosted on WordPress.com? If so, should a user use their custom domain address (i.e. mydomain.com) or their underlying WordPress.com subdomain (i.e. mysite.wordpress.com) address in the “enter url to spider” box? A user posted in the community forums that they tried using it on their site and SF came back with “Connection Refused”. Any help appreci
Hi there,
As per my comment above –
WordPress.com servers respond with a connection refused to a request from the Screaming Frog SEO Spider user-agent.
This appears to be some kind of ‘bot’ related check the server obviously performs, to ensure only real people visit etc.
You can still crawl the websites by changing user-agent (Config > User-agent) and switching to Googlebot, or Chrome etc. But this feature is only available in the licenced version of our software.
Cheers.
Dan
Oh, Great Tutorial! I love your tools screamingfrog! Really Appreciate your developer’s work.
One of the best SEO tool in the market today! awesome tutorial for finding broken links. Keep sharing, Thanks!
We use this tool a lot for onpage seo analysis. Great tool which simplifies otherwise complex processes.
Thanks for sharing this amazing piece of content. I’m using screaming frog and your tool is so amazing.
I have been using this amazing tool for 5+ years and couldn’t do without it!
Great tool, keep developing, I am so excited user of this tool !
Great information, as always, Dan! I use ScreamingFrog almost daily when doing audits on the clients’ website. Don’t know what I would’ve done without it!
Great tool! Very useful to find broken links. Thanks for share.
Having been an avid user of screaming frog for years I thought it about to give some feedback, great tool and fantastic article on how to get the very best out of finding those broken links!
thanks for a great article and an even better bit of kit!
Does your tool craw the internal sites, looking for broken links?
Thank you,
Hi Anselmo,
Yes, it crawls the internal links of sites to find broken links.
It can also find any external links on a site that are broken!
Cheers.
Dan
Hey, your tool seems really interesting to me. Is there a way to get a free/cheap short trial period to test it.
I’d like to use it on checking competitor’s websites for broken links, then contact them to link to one of my other websites that delivers a way higher user-experience.
Best Regards
No doubt screamingfrog provide awesome service. It is one of the best tool recommended by many organization. I also personally recommend it.
Great tool, keep developing, I am so excited user of this tool
Hi, great tutorial and tool… Can you provide information about the license?
Absolutely, you can read more about buying a licence here –
https://www.screamingfrog.co.uk/seo-spider/pricing/
Cheers
Dan
One of the best tools ever, readers must try this!
need more features for finding drop domains in new versions of the program.
Great tool! Very useful to find broken links.
Really helpfull tool.Last year we had a huge amount of broken links. Nowadays only a few
I find this tool a lot easier than the site audit of other tools to check 404s
Previously we used another tool, but using yours definitely saved us a lot of time.
Screaming Frog tool is awesome! Thanks for letting me know how to find broken links.
This one is way better than the other ones I’ve used.
Nice tutorial for checking broken links on a website for SEO.
Screaming Frog has become indispensable in my audit process, thanks
Screaming frog has become one of our most widely used SEO tools.
Screaming Frog is a one of the best tools to search engine optimization. Very simple and useful.
Wouldn’t be able to live without the broken link checker – Screaming Frog is an amazing SEO swiss army knife!
Okay broken links should be fixed, at least important for the user experience, but what I don’t understand is: what is the need for this much noise? (not to mentionin this article,offcourse this is your job). If a link is broken, just skip it. (<— Google) When it comes to SEO, I started to feel like a member of a Help for Search Engines Association. "Great technologies" cannot tolerate little things…
This is really valuable thank-you, finding broken links is crucial and really helps improve the overall performance in SEO. We’ve bounced back and forth between different tools and found this one to be one of the best, if not the best one yet.
The Screaming frog SEO spider tool is very helpful.Thanks for this initiative
Screaming is one of the most widely used SEO tools. I am using this tool till now! It is really superb
Excellent tutorial, I had not seen that there was also this possibility on Screaming Frog. Thanks a lot, now I’ll try it now :)
This is really valuable thank-you, finding broken links is crucial and really helps improve the overall performance in SEO. We’ve bounced back and forth between different tools and found this one to be one of the best, if not the best one yet.
Hi,
best technical seo tool i have ever used. great article likewise.
Useful tool and success tutorial. I’m using free version.
It was enough so far, I’m thinking to pass paid version soon.
Many thanks.
Great article! This software is very useful.
Very helpful tool. I already use it and recommend it to everyone.
Awesome – always using this after a first analysis of a client’s page.
Thank You so much, such a great tool help me a lot finding broken links and am again in competition.
Screaming Frog is an amazing tool that I’ve been using the last 9 years and I never regret using it.
It is the right platform to check the health of a website.
Thank you I really appreciate your help.
thank you very much.deperim geri gaç . easy software
A great tool to check on-page SEO Metrics. Keep up the good word. Team ScreamingFrog Thanks
We use this tool a lot for onpage seo analysis. Great tool which simplifies otherwise complex processes.
Great tool! Thanks a lot.
Works like a charm! Been using it since 2018!
Great stuff! Thanks for the wonderful tutorial. :-)
Thank you I really appreciate your help.
It is a great article on broken link building. I have been using screamingfrog for past two years. It is amazing
Excellent tutorial, I had not seen that there was also this possibility on Screaming Frog. Thanks a lot, now I’ll try it now :)
Great tutorial. Like all other tutorials :-)
Thank you.
Hello, we want to use this for our website wobseo.com, our SEO Agency, but is there a volume licensing? So like 10 different uses?
Licences are per person, you can see volume licencing here (scroll down below the table) – https://www.screamingfrog.co.uk/seo-spider/pricing/
Licences go down to £129 per person from 10.
Cheers.
Dan
I’m using screaming frog for the last 2 years and it’s helped me so much. But this time I found some broken links of my website using Ahrefs but From Screaming Frog it’s doesn’t showing me any broken links. What’s the matter?
Hi Brands Design,
It will be down to configuration. I know Ahrefs crawl ‘nofollow’ links by default for example, we don’t. You need to adjust the config etc.
Feel free to share a real life example via support (support@screamingfrog.co.uk) and we can tell you.
Thanks
Dan
Great tutorial. Like all other tutorials :-)
Great guide, LOVE screaming frog, one of the most used tools in my SEO backpack.
This broken link checker guide using SEO Spider is just awesome. And it works 100%.
I spotted all broken links on my website. Man, this is a great thing.
Brillant tool. Always use it for campaigns.
According to your instructions, I have found the broken links on my website. Thank you
Great instructional tutorial, I had not seen that there was likewise this chance on Screaming Frog. Much appreciated, presently I’ll attempt it now :)
Screaming Frog is a great tool for finding broken backlinks. It is absolutely necessary, especially for digital agencies that provide internal SEO services.
One of the best-broken link checker SEO tools. Amazing post
I really appreciate your tool,it really helps finding broken links of my website. You give a nice tutorial and it helps me a lot working and improve my website, thanks Dan.
Still use screaming frog daily, simply one the best SEO tools for many many years now. Great post.
Thanks for sharing this amazing broken link finder tool really helps a lot
Screaming Frog is by far the best SEO tool I have used.
I’d try this and it works. Very Nice! Thanks.
This is the best SEO tools program in the world. Nice. It is a program that I really recommend.
A must need tool for SEO in their backpack. The broken link checker helped me to increase the ranking and decreased the bounce rate drammatically. If you are using an big website then it’s highly recommended to use the screamingfrog.
Merhaba, Screaming Frog, şimdiye kadar kullandığım en iyi SEO aracıdır.
This tool is a magic wand for e-commerce site. I was trying to fix SEO errors on my website and this came as a handy. This tool simplifies the work. Thanks for the amazing tool that makes life easier for SEO.
we use Screaming Frog in our agency. A really useful SEO Analysis tool.
Screaming Frog is definitely everything that we need as SEO professionals. It always saves time. Thank you screaming frog team for such a detailed blog post & for this amazing tool.
We enjoy using Screaming Frog in our agency. We recommend it to everyone. Thank you for giving us this opportunity.
As Dopinger, we use Screaming Frog with great pleasure. The most reliable tool we use for the satisfaction of our clients.
Thanks for the informative article! Screaming frog has been a great asset to us and we look forward to its continued updates and innovations in the near future. 404’s can be a real hassle!
Great Read
Love screaming frog, it’s proving very, very useful. I found the broken links of my website when checked via the given process.
Tracking and fixing the broken links in the screaming frog is the easiest process though used the first time.
Thank-You for the article.
I really appreciate your tool, it really helps me to find broken links to my website. One of the best tool I have ever used.
Your wonderful article wonderful page FROG IS EXCELLENT I REALLY LIKE IT
Thanks
A super useful tool for searching for broken links. I love it
Great Read
Love screaming frog, it’s proving very, very useful. I found the broken links of my website when checked via the given process.
Tracking and fixing the broken links in the screaming frog is the easiest process though used the first time.
Thank-You for the article.
Still one of my fav tools to use and always will bev:-)
Also a great tool for finding expired domains on large sites :). I even used it to find in stock and out of stock items on suppliers websites to solve a large ecommerce stores stock problem for suppliers that did not have a live inventory feed in this article here>
https://www.ukwebgeekz.com/blog/custom-extraction-screaming-frog/
Really awesome, to find out broken links quicker. Also other functions are useful, thanks.
This is one of the bet tools for not only finding broken links, but also to check that you don’t have duplicate meta titles/descriptions!
I really have no idea how could i do my job without your amazing software, Thank you so much guys for the updates, ill always recommend about you!
As Özyurt Tesisat, we use Screaming Frog with great pleasure. The most reliable tool we use for the satisfaction of our clients.
This is an amazing article, I have used this application for my website, it helps me find out broken links in my website and solved my broken links problems
Thanks
I am in awe of all the possibilities of this tool.
Keep it up!
Hi, great tutorial and tool… Can you provide information about the license?
i was using this before. I am very pleased :)
The best broken link checker there is, period!
Love Screaming frog! It’s very useful for me!
THX
Definitely an awesome feature Dan. Would love to see more Data Studio templates made available for these types of things.
Cheers, Daniel.
Have you seen https://www.screamingfrog.co.uk/how-to-automate-crawl-reports-in-data-studio/?
Hopefully helps!
Dan
Awesome! Thanks
Thanks a lot for this tutorial. I use Screaming Frog for about 2 years and I love it. There are so many features and options for technical analysis and OffPage SEO at the same time. Broken Links are one big issue in the Onsite Audits.
I’ve been using your Broken Link Checker for client sites over the past 7 years but more recently I’ve been using it to find expired domains for personal projects. Top toolkit and needed for every serious SEO.
Helped me to deal with the broken links to my site. Thanks
Great tool! Very useful to find broken links. Thanks for share.
The broken link strategy is one of my favorite linkbuilding strategies.
Because you add value, by improving pages on the web. Thanks for your post!
Great tutorial, we’re using it when introducing SEO junior specialists to the broken links tactics. A must know for every SEO specialist.
Great Tutorial. Seems to be a good Software. Thanks for the post, Dan.
This is a fantastic breakdown. We’ve just handed this over to our new apprentice who picked it very easily by following the guide. Thanks!
Nice post! Thanks for sharing about the broken link checker and it’s really well informative post.
Keep it up.
I love it. For me fixing broken Links & internal redirects is one of the best & fastest ways to improve website quality, user experience and crawl efficiency.
Thanks for this wonderful tool.
Internal linking feature is one of my favourite functionality. It helps in updating the redirecting links which we update in our website or blog.
Thanks for sharing information about the broken link checker and I highly recommended screaming frog tools for SEO optimisation.
Congratulations on this guide. We always use your software that is simple to use but at the same time very complete. Thanks.
Great post Dan, Thanks very helpful Post, I love Screaming Frog and the ability it has. Highly Recommended for all SEO agencies.
Good tutorial. Screaming frog has become one of the most widely used SEO,great topic and best tools for seo
thanks
Hi there – I wish I’d known about Screaming Frog a few years ago!! Argh. Thank you so much for the tutorial. This is going to be invaluable for my website :)
That’s a really useful strategy. SF is an awesome tool for site audits.
We use ScreamingFrog to run audits on the health of our and our clients’ websites. This is our most comprehensive tool for analyzing and improving websites. Thank you!
Definitely, the best on page software so far. I love screaming frog. Cannot live without it :)
I use this tool to find broken links. It is fast and effective. You can easely contact the webites to replace the broken links to get good links for your website.!
I love Screaming Frog, It is hands down the best tool around to find broken links!
I can say that it is the best analysis tool we have used so far. It is very useful for all projects large and small.
Screaming frog is by far the best audit tool on the market!
We use ScreamingFrog to run audits on the health of our and our clients’ websites. This is our most comprehensive tool for analyzing and improving websites. Thank you!
Still use screaming frog daily, been one of the best SEO tools for many years. Great post
Nice tool ! Thanks a lot ! I use every day as a Webmaster and it helps me a lot.
Screamingfrog is like breakfast, can’t start the day planned for technical SEO without it.
We use ScreamingFrog to run audits on the health of our and our clients’ websites. This is our most comprehensive tool for analyzing and improving websites. Thank you!
Screamingfrog is like breakfast, can’t start the day planned for technical SEO without it.
Screaming Frog SEO is by far one of the best & most useful SEO tools!
I have been using it since 2017 and I plan to use it for a long time to come because it is simply irreplacable.
Much greetings from Munich,
Christoph
I love your tutorial. We started using Screaming frog for 3 years. Its a no brainer and one of my favorite tools.
At first it was a lil bit confusing, but after a short time of using i use the different options intuitively. Screaming frog has a lot of features to optimize and identify technical problems of my clients websites. The technical analysis ist comprehensive but totally necessary. I use a lot of the analysis of offpage Seo and broken Links Checker. Thank you for your tutorial.
How to find those URLs which has “#” at the end of the link, please?
e.g- https://bentleypocstg.wpengine.com/software/rail-and-transit/#
https://bentleypocstg.wpengine.com/software/roads-and-bridges/#content
Hi Nitin,
You need to enable ‘Crawl Fragment Identifiers’ under ‘Config > Spider > Advanced’.
Little guide here to find broken bookmarks –
https://www.screamingfrog.co.uk/find-broken-bookmarks/
Cheers
Dan
Great post about broken link checking. I bought a yearly subscription because of this article. Screamingfrog is an amazing product. Keep up the great work on it!
Great tutorial! Without a doubt Screaming Frog is one of the best tools on the market.
This is such a great tool, I’ve been using it for a while now and always incorporate it into my SEO projects, thanks for all the support.
Fantastic tool – I use it basically everyday in order to improve my clients websites.
Screaming Frog is awesome tool. An absolutely “must have” for everyone who works with websites and SEO
I love your tutorial and been using Screaming frog for years. Being able to quickly check all broken links or audit technical errors is a must for all SEOs. Keep up the great work
Thanks for the post, we always use screaming frog.
Great info thanks. Screaming Frog is an essential part of our SEO toolbox.
We would like to say thank you to the Screaming Frog team for such a wonderful tool. This is the most suitable and easy-to-use tool in Search Engine Optimization and helped us a lot. Highly Recommended!
Your tool is the best on the market and this documentation makes it genuinely accessible and invaluable. Thanks for the great work you’ve done and continue to do.
Awesome article. We use Screaming Fog a bunch for fixing technical errors like broken links on our own websites and client websites. Can’t recommend it enough, thanks for sharing!
Great article. There are other ways to leverage Screaming Frog crawler besides broken links. For example, we use it daily to spot patterns in HTML code to produce RSS feeds. We cannt imagine our SEO work without Screaming Frog. You’re the best.
SEO Spider is a great tool and I’ve discovered a few issues with my site and an issue with a previous SSL supplier I thought was resolved. It wasnt
Thank you very much!
Screaming Frog is amazing tool to find broken links. Screaming Frog can also identify other technical issues on your website, such as crawl errors, slow page speed, and duplicate content. Thank you!
SF is the best SEO tool is my go to tool for any website problem audit.
I need a much more detailed video guide to learn how to use all the functions of this unique software. I’d love to see all your resources on this.
Love how flexible SF is at being able to filter the results!
Excellent tutorial! Screaming frog is my go to tool for checking broken links. It is a vital part of my SEO strategy.
Thank you!
And again is SF showing, why its the f*cking best SEO tool on this planet! Thank you for this awesome tutorial!
I think this is still the best tool out there
Thank you so much for providing that tutorial. I found it extremely helpful and was able to successfully repair several broken pages on my website.
Recently bought the licence after being a subscriber to SEO Powersuite for the last 7 years. So glad we finally did the move – the difference in speed and quality is night and day.
What an awesome feature, Dan. Would love to see more Data Studio templates made available for these types of things. I am obliged by seeing what you offered in this post.
Screaming Frog is an essential part of our SEO toolbox. Still the best tool around.
Love screaming frog makes finding bulk 404s a breeze
Great tutorial – question: If the source of a broken link is found to be a redirect in a chain of redirects, how does this impact the process of fixing the broken link? In other words, does the chain of redirects complicate the process, and if so, how can one address this effectively?
Hi Rupert,
If you notice a redirect is the source of the broken link, then use ‘Reports > All Redirects’ and add a filter to the final destination URL response code as a ‘404’.
You’ll then see the source, the redirect URL, and the 4XX all in a single export (you’ll also see any other hops, if there is a chain of redirects).
Cheers.
Dan
It makes finding 404 pages very easy. thanks
It has been a very useful guide on broken links. I will be purchasing a license as soon as possible.
Even after all these years Screaming frog is still our go to tool used in our agency day in day out
Great tutorial! Thank you Screaming Frog. Awesome, to find out broken links quicker.
I’ve been using your SEO tools forever now.. Thanks for the tutorial!
Thanks, man; we use the tool on a bi-monthly base and especially to locate and fix the 404s. Thanks for the tool, and keep up the good work. Especially focus on the speed as well.
A indispensable tool that I have been using for years. Thank you for the development – keep it up
Screamingfrog is like breakfast, can’t start the day planned for technical SEO without it.
I have been utilizing ScreamingFrog for auditing my own sites. Recently I stumbled upon a very good amount of PDF files that contain links. However, checking every link is a bit difficult for me to go through and see If they are working or broken. Is there any way I can use Screamfrog or any other tool to check broken links in PDF files?
Thank You
Great tutorial, thank you very much. An essential tool !
great tutorial, I love screaming frog and with your tutorial even more :)
Thanks!!
Theres no doubt about it Screaming Frog is our go to SEO tool for any webpage audit. We have been using this SEO Tool for over 8 years, and the best thing about it is, it us just constantly getting better, new features, better design, more integrations. It can be used for so many SEO tasks, its a MUST for any SEO agency.
It’s great to see the SEO Spider tool in action, but it also reminds me of the power of web scraping for data extraction and analysis. Both SEO and web scraping rely on leveraging data for optimization, and it’s fascinating to see how technology can help us in various aspects of online management. This article is a testament to the importance of staying updated with the latest tools and techniques in the digital world.
Managing 404 errors effectively is essential for SEO, in my experience. I’ve noticed that broken links not only harm user experience but also affect search engine rankings. A unique advantage of Screaming Frog, I find, is its thorough and accurate analysis. It identifies both internal and external broken links, which I consider essential for the overall health of your website.
What I particularly appreciate about Screaming Frog is its ability to detect ‘jump links’. This, I believe, improves the navigation and accessibility of your site, benefiting the user experience. I find the flexible export options very useful as they make it easy to analyze and share data, especially beneficial for larger sites or team projects.
In my opinion, regularly checking and correcting broken links with a tool like Screaming Frog is an indispensable part of an effective SEO strategy.
Fantastic post! We greatly appreciate the insights shared on how to find broken links using the SEO Spider. Keep up the great work!