Just a quick post to let you all know we have released an update of the Screaming Frog SEO spider to version 2.10.

screaming-frog-seo-spider-2-10

The updated version of the SEO spider has plenty of new features, which include –

  • Save Configuration – You can now save the spider configuration to be default on start-up. This feature is fairly basic currently, but will be developed further to allow a number of configurations to be saved and accessed quickly.
  • Response Time – The SEO spider now collects response times for URLs under the ‘Response Codes’ tab. This should help identify slow loading pages!
  • Filter Totals – There is now a filter total in the GUI, to save exporting to Excel and calculating.
  • Bulk Export Out Links – You can now bulk export out link data via the ‘advanced export’ menu.
  • Accept Cookies – The SEO spider can now accept cookies, which can be useful for some websites which require it to be able to crawl. This feature is still turned off as default as of course, search bots do not accept cookies.
  • Auto Pause On Reaching Memory – One of our most frequent queries is, how many URLs can the SEO spider crawl? Well, this is dependent on memory allocation. Now when you start to reach the memory allocation, the SEO spider will automatically pause and recommend saving the crawl and increasing memory allocation, before uploading the crawl again and resuming.
  • 5XX Response Retries – There is now an ‘advanced’ tab in the spider configuration which allows the user to adjust the number of automatic retries for 5XX responses.
  • Request Authentication – Users now have the ability to switch off messages from websites which require authentication.
  • Response Timeout – The amount of time the SEO spider waits for a response from a URL is now configurable, which is useful for very slow loading websites.
  • New Filters – We have introduced a couple of new filters, such as page titles with under 30 characters or meta descriptions with under 70 characters.

You can download the new version on our SEO spider page.

Update 2.11 (25th February 2013)

We have released a small update to version 2.11. This release essentially irons out a few bugs in version 2.10 which include –

  • Fixed a ‘Null Pointer Exception’ error when exporting the ‘response codes’ tab for any results where the response time field has no value (such as a DNS look-up failure).
  • Fixed a bug with custom filters loading from older projects.
  • Fixed a domain matching bug with .com and .com.au’s.
  • Changed the date format in the SEO spider so there is no confusion between UK and US date formats :-).

There were a number of other smaller amends and updates along the way as well. As always, if you spot any problems, please contact us via the support page with the details.

Thanks again to everyone for all their support of the tool, we are already working on the next batch of features. There are still plenty on the list!

screamingfrog (31 Posts)

Dan Sharp is founder & Director of Screaming Frog. He has developed search strategies for a variety of clients from international brands to small and medium sized businesses and designed and managed the build of the innovative SEO Spider software.