Log File Analyser FAQ

Why do I get “error opening file for writing” when installing?

Try running the file as administrator by right clicking the installer and choosing “Run as administrator”. Alternatively log in to an administrator account. You may need to request assistance from your IT department depending on your company setup.

Back to top

Why can’t my Licence Key be saved?

The SEO Spider stores the licence in a file called licence.txt in the users home directory in a ‘.ScreamingFrogSEOSpider’ folder. You can see this location by going to Help->Debug and looking at the line labeled “Licence File”. Please check the following to resolve this issue:

  • Ensure you are able to create the licence file in the correct location.
  • If you are using a Mac, see the answer to this stackoverflow question.
  • If you are using Windows is could be the default user.home value supplied to Java is incorrect. Ideally your IT team should fix this. As a work around you can add: -Duser.home=DRIVE_LETTER:\path\to\writable\directory\ to the ScreamingFrogSEOSpider.l4j.ini file that controls memory settings.

Back to top

What Log File formats are supported?

The Log File Analyser supports Apache, W3C and Amazon Elastic Load Balancing formats.

Apache

The following fields must be present:

  • Time the request was received: %t
  • First line of request: %r
  • Status: %>s
  • User Agent: %{User-agent}i
The following are optional:
  • Referer: %{Referer}i
  • Size of response in bytes: %b
  • Remote hostname: %h
The Log File Analyser is able to determine any format that provides these required fields. All these fields are present in the NCSA extended/combined log format, which looks like this:

"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

Please note that Common Log Format (CLF) is not supported, as it does not provide a User Agent.

Please contact support if your log cannot be read.

W3C

The following fields must be present:
  • date
  • time
  • cs-uri-stem
  • sc-status
  • cs(User-Agent)
The following are optional:
  • cs-uri-query
  • cs(Referer)
  • time-taken
  • sc-bytes
  • cs-method
  • c-ip
Please contact support if your log cannot be read.

Amazon Elastic Load Balancing

This format can't be configured, it's a fixed format which the Log File Analyser supports.

Please contact support if your log cannot be read.

Back to top

Why does the Log File Analyser open then immediately close?

This is a known java 8 bug relating to fonts. You can see details of this bug here.

The issue is around the use of non-standard fonts. If you restore standard fonts the Log File Analyser will be able to start. To do this, open the “Font Book” application and choose File->Restore Standard Fonts. The Log File Analyser should now be able to start without issue.

The removed fonts will now appear in the folder /Library/Fonts (Removed). You can then add them back in one by one, so you can work out and avoid the one(s) that are causing the issue.

Back to top

Do you work with resellers?

Resellers can purchase an Log File Analyser licence online on behalf of a client. Please be aware that licence usernames are automatically generated from the first and last names used in the Billing Address entered during checkout. If you require a custom username, then please request a PayPal invoice in advance. For resellers who are unable to purchase online with PayPal or a credit card and encumber us with admin such as vendor forms, we reserve the right to charge an administration fee of £50.

Back to top

What is the part number?

There is no part number or SKU.

Back to top

Where can I get company information?

On our contact page.

Back to top

Where can I get Form W-9 information?

Screaming Frog is a UK based company, so this is not applicable.

Back to top

How do I buy a licence?

Simply click on the ‘buy a licence’ option in the Log File Analyser ‘licence’ menu or visit our purchase a licence page directly. You can then create an account & make payment. When this is complete, you will be provided with your licence key to open up tool & remove the crawl limit. If you have just purchased a licence and have not received your licence, please check your spam / junk folder. You can also view your licence(s) details and invoice(s) by logging into your account.

Back to top

Do you offer discounts on bulk licence purchases?

Yes, please see our Log File Analyser licence page for more details on discounts.

Back to top

Do you accept payment via American Express?

Not directly, but by selecting the “PayPal” option on our Billing page: amex Then choosing “Check Out as Guest” when you get to PayPal you should be able to use your Amex card. This option is available in most countries. If you don’t get this option, please use another card, PayPal or contact us via support and you can set-up to pay via bank transfer.

Back to top

I have purchased a licence, why have I not received it?

If you have just purchased a licence and have not received your licence, please check your spam / junk folder. Licences are sent immediately upon purchase. You can also view your licence(s) details and invoice(s) by logging into your account. Please also check your payment method. If you have paid via an e-cheque, then the licence will only be sent when it has cleared. Paypal explains this as well.

Back to top

Why is my credit card payment being declined?

There are a few reasons this could happen:

  • Incorrect card details: Double check you have filled out your card details correctly.
  • Incorrect billing address: Please check the billing address you provided matches the address of the payment card.
  • Blocked by payment provider: Please contact your card issuer. Screaming Frog does not have access to failure reasons. It’s quite common for a card issuer to block international purchases.

Back to top

What additional features does a licence provide?

A licence removes the 1,000 line log file limit and allows you to have multiple projects saved at a time. We also provide support for technical issues related to the Log File Analyser for licensed users. Licences are individual per user. If you have five members of the team who would like to use the licenced version, you will need five separate licences for each user.

Back to top

Can I use my licence on more than one device?

Yes. The licence allows you to install the Log File Analyser on multiple computers. Licences are individual per user. Please see section 3 of our terms and conditions for full details.

Back to top

How do I submit a bug / receive support?

Please follow the steps on the Log File Analyser support page so we can help you as quickly as possible. Please note, we only offer full support for premium users of the tool although we will generally try and fix any issues.

Back to top

How do I provide feedback?

Feedback is welcome, please just follow the steps on the support page to submit feedback. Please note we will try to read all messages but might not be able to reply to all of them. We will update this FAQ as we receive additional questions and feedback.

Back to top

Where are my projects stored?

This varies depending on operating system, please replace USERNAME with the correct username for the operating system:

  • ​Windows: C:\Users\USERNAME\.ScreamingFrogLogfileAnalyser\projects\
  • ​Mac: /Users/USERNAME/.ScreamingFrogLogfileAnalyser/projects/
  • Linux: /home/USERNAME/.ScreamingFrogLogfileAnalyser/projects/

Back to top

Why are the numbers different to Google Search Console?

  • Google Search Console (GSC) numbers are for all Google User Agents, the numbers shown in the Log File Analyser are on a per User Agent basis, and don't include all of Google's.
  • GSC figures are calculated using Pacific Daylight Time (PDT) however the Log File Analyser uses UTC by default. UTC is 7 hours ahead of PDT. You can adjust the UTC offset to match PDT by going to Project->Settings and choosing UTC-07:00 from the dropdown.
  • 'Pages crawled per day' in GSC is all requests. If the same URL is requested 10 times, this is regarded as 10 pages.
  • Your log might be from a web server that sits behind a cache, so you're not seeing all the requests.
  • Your site might be load balanced across multiple servers and you've not imported logs from all the servers.

Back to top

Why do I receive a 'protected your PC' message on installation in Windows 10?

When you install the Log File Analyser on Windows 10, you may receive a 'protected your PC' SmartScreen - Windows 10 SmartScreen To install, you have to click the 'More Info' link underneath and then 'Run Anyway' - smart screen more info link The reason for this is the certificate is new and it takes time for this message to disappear, despite being registered Windows developers.

Back to top

Why can't I import CLF format Apache Log File?

The Apache Common Log Format (CLF) is usually the default format configured for many different web servers.

Unfortunately this format does not provide a User Agent field which is an integral part of the analysis provided by the Log File Analyser.

You will have to reconfigure or ask the administrator of the web server to reconfigure the log format string to provide the user agent field.

Here is the typical NCSA log format line required:

"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

Back to top

What happens when the licence expires?

When the licence expires, the Log File Analyser returns to the restricted free lite version. You will be able to open old projects, but you won't be able to add to them or create new projects.

Back to top

How do I send you debug logs?

You can get the debug logs from the Log File Analyser by going to Help->Debug->Save Logs. This will allow you to save a zip file containing the logs we need to help diagnose any issues.

If for some reason you are unable to start the Log File Analyser locate and attach the trace.txt file. This is found in a different location depending on your OS. Please substitute USERNAME for your own username.

Windows:
C:\Users\USERNAME\.ScreamingFrogLogfileAnalyser\trace.txt

OS X & Linux:
~/.ScreamingFrogLogfileAnalyser/trace.txt

Back to top

What's the maximum supported Log File size?

The Log File Analyser uses a database to store the data. So the volume of logs that can be stored will be limited by the size of your hard drive. Performance will also be limited to how fast your disk is - an SSD will give you greater performance than a spinning disk. The size of your site is not a relevant factor, it's how much data you have in your log files. A small but busy site could have a lot more log events than a large site that doesn't get much traffic. We often import several Gigabytes worth of uncompressed log files without issue. The larger the time frame you request data for, the more events that have to be considered, and the slower the response will be.

Back to top

Can I use my SEO Spider licence in the Log File Analyser?

No, the Screaming Frog SEO Spider is a separate product to the Log File Analyser. They have different licences, which will need to be purchased individually.

Back to top

How is the software delivered?

The software needs to be downloaded from our website, the licence key is delivered electronically by email.

Back to top

What is the reseller price?

We do not offer discounted rates for resellers. The price is GBP at £99 per year, per user.

Back to top

Where can I get licensing terms?

Licensing details can be found here.

Back to top

Can I get a quote in a currency other than GBP?

No, we only sell in GBP.

Back to top

How much does the Screaming Frog Log File Analyser cost?

As standard you download the free version of the tool. However, without a licence the Log File Analyser is limited to analysing 1,000 log file event lines in a single project. For £99 per annum you can purchase a licence which removes the 1,000 log file event limit and number of projects you can create. A licence is required per individual using the tool. When the licence expires, the Log File Analyser returns to the restricted free version.

Back to top

What payment methods do you accept & from which countries?

We accept PayPal and most major credit and debit cards. Payment via American Express is covered here. The price of the Log File Analyser is in pound sterling (GBP). If you are outside of the UK, please take a look at the current exchange rate to work out the cost. (The automatic currency conversion will be dependent on the current foreign exchange rate and perhaps your card issuer). We do not accept cheques (or checks!)

Back to top

How do I renew my licence?

At the moment the best way is to simply purchase another licence upon expiry. Licences do not auto renew – so if you do not want to renew your licence you will not be charged and need to take no action.

Back to top

I’m a business in the EU, can I pay without VAT?

Yes. To do this you must have a valid VAT number and enter this on the Billing page during checkout. Select business and enter your VAT number as shown below: enter_vat_number Your VAT number will be checked against the VIES system and VAT removed if it is valid. The VIES system does go down from time to time, so if this happens please try again later. Unfortunately we cannot refund VAT once a purchase has been made.

Back to top

Do you have a refund policy?

If you are not completely satisfied with the Log File Analyser software you purchased from this website, you can get a full refund if you contact us within 14 days of purchase. To obtain a refund, please follow the procedure below.

Contact us via lfasupport@screamingfrog.co.uk or support and provide the following information:

  • Your contact information (last name, first name and email address).
  • Your order number.
  • Your reason for refund! If there's an issue, we can generally help.
  • For downloaded items, please provide proof that the software has been uninstalled from all your computers and will never be installed or used any more (screenshots will suffice).
If you have purchased your item by credit card the refund is re-credited to the account associated with the credit card used for the order.

If you have purchased your item by PayPal the refund is re-credited to the same PayPal account used to purchase the software.

If you have purchased your item using any other payment method, we will issue the refund by BACS, once approved by our Financial Department.

For any questions concerning this policy, please contact us at support.

Back to top

How many users are permitted to use one licence?

Licences are individual per user. A single licence key is for a single authorised user. If you have five users using five copies of the Screaming Frog Log File Analyser software, you will require 5 separate licences. Please see section 3 of our terms and conditions for full details.

Back to top

Why is my Licence Key saying it’s invalid?

If the Log File Analyser says your ‘licence key is invalid’, then please check the following, as the licence keys we provide always work.

  • Ensure you are using the username we provided for your licence key, as this isn't always the same as your account username. This is by far the most common issue we see.
  • Copy and paste the username and licence key, they are not designed to be entered manually.
  • Please also double check you have inserted the provided ‘Username’ in the ‘Username’ field and the provided ‘Licence Key’, in the ‘Licence Key’ field.
  • You are entering a SEO Spider licence into the Log File Analyser.
  • You are entering a Log File Analyser licence into the SEO Spider.
  • If your licence key still does not work, then please contact support with the details.

    Back to top

Why does the Log File Analyser not import my log file?

The most likely reason is the file you are importing is not a log file, or more specifically, an access log file. Please read our introduction to log files guide that gives examples of the supported log file formats so you can compare with your own.

Common formats imported that are not log files are:

  • CSV files
  • JSON files
  • Apache error logs
The Log File Analyser supports log files in W3C and Apache format. These logs are produced by the 3 most popular web servers: Apache, IIS and Nginx. There is no need to specify what format your logs are in, the Log File Analyser will auto detect this when you upload them.

If you have any problems importing logs, then please contact our support team.

Back to top

Why won’t the Log File Analyser start?

This is nearly always due to an out of date version of Java. If you are running the PC version, please make sure you have the latest version of Java. If you are running the Mac version, please make sure you have the most up to date version of the OS which will update Java. Please uninstall, then reinstall the Log File Analyser and try again.

Back to top

Why am I seeing inconsistent response codes for a URL?

The Screaming Frog Log File Analyser is obviously analysing data over a period of time, directly from the server logs. So a URL might have been broken historically and subsequently been fixed, and hence why it has different or 'inconsistent' responses. Another important thing to remember is, if you have uploaded a log file with relative URLs and had to provide the site URL on import, the non www. and www. version of URLs will be aggregated. So events to https://screamingfrog.co.uk and https://www.screamingfrog.co.uk would be aggregated to https://www.screamingfrog.co.uk if that's the address provided upon import. The non www. version might be set-up to 301 redirect to the www. version, and hence, half the responses redirect, while the other half provide 200 responses. This is why we generally recommend logs are configured with absolute URLs. However, you may also find that a server is delivering inconsistent responses when under load.

Back to top

Why can't I see all traffic?

By default the Log File Analyser only stores events from known bots. This minimises the number of events imported and speeds up analysis. If you would like to see all events, untick the "Store Bot Events Only (Improves Performance)" option when creating a new project. This option can only be set when creating a project. bots_only

Back to top

How do you handle logs with multiple protocols and/or domains?

If you have an access log that contains entries for both http and https traffic, but the lines don't include the protocol, the Log File Analyser has no way of knowing which lines are for http and which are for https. The same is true if you have an access log that contains entries for multiple domains. Theses are both very much log file configuration issues. If no protocol and/or domain is specified, you'll be asked for it.

To include the protocol and domain for Apache style logs, the request line should be an absolute url. So rather than:

[01/Jan/2017:00:00:01 +0000] "GET /seo.html HTTP/1.1" 200 "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

The full protocol and domain will be on each line:

[01/Jan/2017:00:00:01 +0000] "GET http://example.com/seo.html HTTP/1.1" 200 "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

This way the Log File Analyser can clearly see the protocol and domain for each line.

For W3C logs, you'll need to include the cs-protocol (Advanced Logging needed for IIS) field along with either cs-host or x-host-header.

Back to top