Screaming Frog SEO Spider Update – Version 2.50

Dan Sharp

Posted 1 July, 2014 by in Screaming Frog SEO Spider

Screaming Frog SEO Spider Update – Version 2.50

I am pleased to announce a small update of the Screaming Frog SEO spider to version 2.50, codenamed internally as ‘bite’. Thanks again to everyone for their continued feedback, suggestions and support.

This is just a small update, with a couple of new features and bug fixes listed below. Our next update will be much, much larger. Version 2.50 includes just the one main new feature –

Graph View

We have introduced a new ‘graph view’ window into the lower half of the right hand overview window pane. This updates in real time, as you crawl –

seo spider graph view

When you click the different tabs at the top, or filters in the right hand window pane, the graphs update to help visualise the data.

seo spider graph view again

If you click the ‘Site Structure’ or ‘Response Codes’ tabs, you can see better visualisations of this data as well.

seo spider graph view again, again

At the moment you can’t export the graphs unfortunately, but we do have this planned for development.

Other Updates

We have also made a number of other updates, these include the following –

  • We became a verified publisher for the Windows platform.
  • Improved debugging and saving of log files for support issues directly from the ‘Help’ and ‘debug’ menu.
  • Improved logging so data is not overwritten with new sessions.
  • The SEO Spider now detects any crashes on a previous run and asks the user to save logs and send to our support team.
  • The SEO Spider now has greater tolerance around invalid gzip responses. We will now continue with what was read up to the point of error, logging appropriately in the log file.
  • Fixed a bug where the ‘ignore robots.txt’ configuration wouldn’t save as default.
  • Fixed a bug where the ‘respect canonicals’ configuration was case insensitive.
  • Fixed a bug where we failed to parse some pages containing meta refresh tags resulting in invalid urls.
  • Fixed a crash caused by the sequence add, add, delete, delete, add on URL Rewriting -> Remove Parameters config menu :-).
  • Fixed a crash caused by starting in crawl mode, navigating to the outlinks tab and right clicking.

That’s the lot for this fairly small update. We still have plenty more in the pipeline and our next version will be a much larger release!

Thanks for all your support and feedback as always, you can download the new version now.

Move to Java 7

Version 2.50 requires Java 7 to use (rather than Java 6), which was a natural evolution and launched around three years ago now.

It’s also worth noting that Macs below OS X Version 10.7.3 (& 32-Bit Macs) can continue to use version 2.40, as well as anyone who might be experiencing any bugs in the new version.

Dan Sharp is founder & Director of Screaming Frog. He has developed search strategies for a variety of clients from international brands to small and medium-sized businesses and designed and managed the build of the innovative SEO Spider software.

31 Comments

  • Guilherme 10 years ago

    Great update, really useful in reports.

    I have a suggestion, why don’t you add a filter to show if any page has the Google Analytics script missing?

    Best and continue with the great work!

    Reply
    • screamingfrog 10 years ago

      Hi Guilherme,

      Thanks for the kind comments and for the suggestion.

      We do have this on our ‘todo’, obviously you can do it manually at the moment via the custom filters, but we were going to introduce something that helps do this automatically / easier :-)

      Thanks again,

      Dan

      Reply
  • Richard 10 years ago

    The last 3 or 4 updates have been great!

    Reply
  • Guillaume 10 years ago

    Hi Guys,
    Thanks for that great update !

    Don’t worry for graph exports, in the meantime, we will do screen captures.

    Keep up that good work :)

    Reply
  • Eric 10 years ago

    Hi
    great news, your product is very efficient
    but for me there is somthings missing yet !
    :)
    it ‘s will be very funny if we can know where you can find easily the external links on the report

    best regards

    Reply
    • screamingfrog 10 years ago

      Hi Eric,

      Apologies if I don’t understand the question completely.

      You can view ‘external links’ under the ‘external’ tab and you can view where a site links out to these, by clicking on the ‘in links’ tab at the bottom.

      If you mean, inbound links (ie links coming URLs found in the crawl), this is on our ‘todo’ for the future.

      Thanks,

      Dan

      Reply
  • Adam Heaton 10 years ago

    We are spoilt with these updates, can’t wait for the coming updates! Wonder what else you guys have in store for us!

    Reply
  • Mike Cravens 10 years ago

    Would you please consider continuing to ship binaries for Intel Macs running Intel CoreDuo (32 bit address) processors.

    In the latest version of Apple’s Xcode developers’ package they default to not supporting this machines, reversing a long standing Apple policy of defaulting to include binaries for practically every machine Apple ever made in standard Universal Binaries.

    The Core Duo processor Imacs are fairly recent and are still in use at my customers, in schools, and in our offices and homes.

    I promise to upgrade our licenses and to continue to promote your software and services in the seminars we teach if you will only change back your Xcode setup to what is was until just recentl and ship Intel 32 bit Core Duo binaries along with 64 bit binaries.

    Thank you for such a wonderful tool. I hope to be able to continue using and promoting it.

    Regards,

    Mike Cravens
    Managing Partner
    Southwest Digital Media

    Reply
    • screamingfrog 10 years ago

      Hi Mike,

      Thanks for your comment (and the email you popped through).

      The team have just responded to you via email and I have put together an FAQ here –

      https://www.screamingfrog.co.uk/seo-spider/faq/#45

      Essentially, 32-bit Macs are still supported for version 2.40 which you can continue to download and use.

      2.50 requires newer technology only availble in Java 7 which is not available for OS X below 10.7.3 (& 32-Bit Macs). The difference, is really just the graphs.

      Cheers.

      Dan

      Reply
  • Jared Kayes 10 years ago

    Another good update I see.

    Reply
  • Matthew 10 years ago

    Great update. May I suggest implementing export graphs to jpeg or png functionality. This would make client reporting a lot easier for me. Keep up the good work.

    Reply
    • screamingfrog 10 years ago

      Hi Matthew,

      Thanks for the suggestion.

      I mentioned in the post we will be introducing this functionality definitely.

      Cheers.

      Dan

      Reply
      • Matthew 10 years ago

        My bad. Again, thanks for the great work and support – very impressive tool

        Reply
      • matthew 10 years ago

        Hi Dan,

        I’ve having issues with Screamingfrog on my mac. It hangs badly whenever I use it. I’m running an I7 MBP running 10.9.4 any suggestions or known bugs?

        Reply
  • Ian McGarvey 10 years ago

    Great update. I wanted suggest / request a new feature (First of all, a feature request page / forum would be nice):

    I work at a website company and one of the things on our pre-launch checklist is to confirm that all external links open in a new window/tab.

    It would be awesome if this software could capture the “target” attribute of links so that I could sort external links by that column and see what external links are not target=”_blank.”

    Thanks for your time and effort in general and in response to this request.

    Reply
    • screamingfrog 10 years ago

      Hey Ian,

      Thanks for your suggestion, I can confirm we do have this on the ‘todo’.

      I do like the idea of a feature request page as well. We have been considering this too!

      Cheers,

      Dan

      Reply
      • Marc 9 years ago

        Did you guys ever get around to doing this?
        Is there any other way to effectively extract this information?

        Reply
  • Josh 10 years ago

    Hi,
    i have a problem with the Graph View.
    It works in the first time, but not now.

    I uninstalled the software and reinstalled it.
    This does not work, I cant see the graph.
    The area does not exist anymore.

    What can I do?

    Reply
    • screamingfrog 10 years ago

      Hey Josh,

      It sounds like you just need to click the little ‘arrow’ in the top right, which makes the right hand overview window open up.

      OR if the right hand window is open and just the ‘graph view’ is missing, you might need to click the little ‘arrow’ at the bottom.

      You can see them in the screenshots above.

      Cheers,

      Dan

      Reply
  • Jim Hodson 10 years ago

    Like the new visualization.

    Was disappointed that the bug related to “Configuration -> Spider -> Crawl all subdomains” was not fixed. It existed in the last release and exists in this one.

    When checked, it essentially causes the program to crawl ALL linked domains and subdomains (internal and external), crawl all links on those external domains, ad infinitum.

    I thought if checked and crawling http://example.com/, it instructed to the program to crawl all subdomains of example.com (current domain being crawled). But instead it acts like “crawl all linked domains” or “crawl the web” lol.

    Reply
    • screamingfrog 10 years ago

      Hi Jim,

      Thanks, excellent to hear you like!

      Sorry you’re disappointed we didn’t fix this bug in this update. It is known and on our list, but actually wasn’t a priority as it’s easy to work around.

      If you’re crawling from the root (for example http://example.com/), the SEO spider will crawl all subdomains as default anyway. So there isn’t actually any need to use the ‘Crawl all subdomains’ option, which is when you correctly pointed out this bug occurs.

      This is similar to a site: query in Google, where Google will show all subdomain URLs in their index if you query the root. If you specify a subdomain in the site: query, it will only show that subdomain results etc.

      If you’re crawling from a subdomain, like http://www.example.com/, you can still use the ‘Crawl all subdomains’ configuration fine.

      Hopefully that helps for now, we will fix it though!

      Cheers,

      Dan

      Reply
  • Matt Storms 10 years ago

    Increasing the size of the arrows to see the graph view would be helpful or accent them with a color.

    Reply
    • screamingfrog 10 years ago

      Hey Matt,

      Thanks for the comment & follow up tweet.

      The issue is a bug, where the right hand window doesn’t open for the first time as default (so some don’t realise it’s even there).

      The arrow improvements would help in terms of UX, but we have now fixed the bug too :-)

      Cheers.

      Dan

      Reply
  • Emin Sinanyan 10 years ago

    Love the new version. The graphs are amazing. I have one suggestion. When the tool scans for a url, it would be nice to highlight the errors. For example 404 errors red, 301s yellow or some other color. It would be easier on the eye for huge sites. Thank you!

    Reply
    • screamingfrog 10 years ago

      Hey Emin,

      Thank you, very kind.

      The colours of the graphs/charts have been chosen with that in mind already.

      For example for response codes, the colours are as follows –

      No Response – Grey
      2XX – Green
      3XX – Orange
      4XX – Red
      5XX – A dark red.

      We found this to quite nice to help spot obvious problems. Hope that helps :-)

      Dan

      Reply
  • othmane 10 years ago

    Hello,

    Great update 2.5 , i would suggest to add average depth like average response time it’s would help to know average depth for the website.

    Also i recommande when we click on the nombre of uri in depth (graph view) we can have acces to this url without having to export to excel.

    Cheers.
    Othmane

    Reply
    • Bruno Pinna 9 years ago

      hey team of Screaming Frog
      thank you for develop this tool because my life now is more easy
      really i doing on hand… your tool save my time.
      great job but i want news update :)
      cheers from brazil seo

      Reply

Leave A Comment.

Back to top