Log File Analyser FAQ

Licencing
Why is my licence key invalid?

If the Log File Analyser says your ‘licence key is invalid’, then it's because it has been entered incorrectly. This can be seen under 'Licence > Enter Licence Key' in the interface.

Licence keys are displayed on screen when you check out, sent in an email with subject "Screaming Frog Log File Analyser licence details" and available at any time by logging into your account.

Before contacting us, please view your licence details and copy and paste them into the correct field - they are not designed to be entered manually. Please follow our tips below on common issues encountered.

  • Ensure you are using the username we provided for your licence key, as this isn't always the same as your account username and it's not your email address. This is by far the most common issue we see
  • Copy and paste the username and licence key, they are not designed to be entered manually.
  • Please also double check you have inserted the provided ‘Username’ in the ‘Username’ field and the provided ‘Licence Key’, in the ‘Licence Key’ field.
  • You are entering a SEO Spider licence into the Log File Analyser.
  • You are entering a Log File Analyser licence into the SEO Spider.

If your licence key still does not work, then please contact support with the details.

How many users are permitted to use one licence?

Licences are individual per user. A single licence key is for a single authorised user. If you have five users using five copies of the Screaming Frog Log File Analyser software, you will require 5 separate licences. Please see section 3 of our terms and conditions for full details.

Is it possible to use a licence on more than one device?

Yes. The licence allows you to install the Log File Analyser on multiple computers. Licences are individual per user. Please see section 3 of our terms and conditions for full details.

What happens when the licence expires?

When the licence expires, the Log File Analyser returns to the restricted free lite version. You will be able to open old projects, but you won't be able to add to them or create new projects.

Why can’t my licence key be saved (Unable to update licence file)?

The Log File Analyser stores the licence in a file called licence.txt in the users home directory in a ‘.ScreamingFrogLogfileAnalyser’ folder. You can see this location by going to Help->Debug and looking at the line labeled “Licence File”. Please check the following to resolve this issue:

  • Ensure you are able to create the licence file in the correct location.
  • If you are using a Mac, see the answer to this stackoverflow question.
  • If you are using Windows is could be the default user.home value supplied to Java is incorrect. Ideally your IT team should fix this. As a work around you can add: -Duser.home=DRIVE_LETTER:\path\to\writable\directory\ to the ScreamingFrogLogfileAnalyser.l4j.ini file.

Can I use my SEO Spider licence in the Log File Analyser?

No, the Screaming Frog SEO Spider is a separate product to the Log File Analyser. They have different licences, which will need to be purchased individually.

I have purchased a licence, why have I not received it?

If you have just purchased a licence and have not received your licence, please check your spam / junk folder. Licences are sent immediately upon purchase. You can also view your licence(s) details and invoice(s) by logging into your account. Please also check your payment method. If you have paid via an e-cheque, then the licence will only be sent when it has cleared. Paypal explains this as well.

What additional features does a licence provide?

A licence removes the 1,000 line log file limit and allows you to have multiple projects saved at a time. We also provide support for technical issues related to the Log File Analyser for licensed users.

Licences are individual per user. If you have five members of the team who would like to use the licenced version, you will need five separate licences for each user.

Where can I find your EULA (terms & conditions)?

You can read our EULA here.

Billing
How do I buy a licence?

Simply click on the ‘buy a licence’ option in the Log File Analyser ‘licence’ menu or visit our purchase a licence page directly.

You can then create an account and make payment. When this is complete, you will be provided with your licence key to open up tool and remove the log file and project limits. If you have just purchased a licence and have not received your licence, please check your spam / junk folder. You can also view your licence(s) details and invoice(s) by logging into your account.

Please note, if you purchased via bank transfer or PayPal invoice you will not have an account. An account is only created when you checkout via our online purchase system. If you have lost any of your details you can contact us for this information.

How do I renew my licence?

You simply login to your existing account and purchase another licence upon expiry.

Please don't purchase a licence pre-expiry as it will not extend your existing licence expiry. Updated licence details will be provided upon purchase to continue using the tool.

Licences do not currently auto renew, so if you do not want to renew your licence you need to take no action as you will not be automatically charged.

How much does the Screaming Frog Log File Analyser cost?

As standard you download the free version of the tool. However, without a licence the Log File Analyser is limited to analysing 1,000 log file event lines in a single project.


For £99 per annum you can purchase a licence which removes the 1,000 log file event limit and number of projects you can create.


A licence is required per individual using the tool. When the licence expires, the Log File Analyser returns to the restricted free version.

Do you offer licence discounts?

We only provide discounts on bulk purchases of 5 or more licences at a time. We do offer discounts at any other time.


We simply keep the price as competitive as possible for everyone.


Please see the Log File Analyser licence page for more details on discounts.

What payment methods do you accept & from which countries?

We accept most major credit and debit cards and PayPal.

The price of the SEO Spider is in pound sterling (GBP), but it can be purchased worldwide.

If you are outside of the UK, please take a look at the current exchange rate to work out the cost. The automatic currency conversion will be dependent on the current foreign exchange rate and perhaps your card issuer, so we are not able to give an exact fee (other than in GBP).

We do not accept cheques (or checks!) as payment.

I’m a business in the EU, can I pay without VAT?

Yes. To do this you must have a valid VAT number and enter this on the Billing page during checkout. Select business and enter your VAT number as shown below: enter_vat_number Your VAT number will be checked against the VIES system and VAT removed if it is valid. The VIES system does go down from time to time, so if this happens please try again later. Unfortunately we cannot refund VAT once a purchase has been made.

Why is my credit card payment being declined?

Unfortunately we do not get to see why a payment has been declined. There are a few reasons this could happen:

  • Incorrect card details: Double check you have filled out your card details correctly.
  • Incorrect billing address: Please check the billing address you provided matches the address of the payment card.
  • Blocked by payment provider: Please contact your card issuer. Screaming Frog does not have access to failure reasons. It’s quite common for a card issuer to automatically block international purchases.

The most common of the above is an international payment to the UK. So please contact your card issuer and ask them directly why a payment has been declined, and they can often authorise international payments.


Alternatively, you can try a different card, PayPal, or you can contact us via support and pay via bank transfer.

Do you have a refund policy?

Absolutely! If you are not completely satisfied with the Log File Analyser you purchased from this website, you can get a full refund if you contact us within 14 days of purchasing the software. To obtain a refund, please follow the procedure below.

Contact us via lfasupport@screamingfrog.co.uk or support and provide the following information:

  • Your contact information (last name, first name and email address).
  • Your order number.
  • Your reason for refund! If there's an issue, we can generally help.
  • For downloaded items, please provide proof that the software has been uninstalled from all your computers and will never be installed or used any more (screenshots will suffice).

If you have purchased your item by credit card the refund is re-credited to the account associated with the credit card used for the order.


If you have purchased your item by PayPal the refund is re-credited to the same PayPal account used to purchase the software.


If you have purchased your item using any other payment method, we will issue the refund by BACS, once approved by our Financial Department.


For any questions concerning this policy, please contact us at support.

Resellers
Do you work with resellers?

Resellers can purchase an Log File Analyser licence online on behalf of a client. Please be aware that licence usernames are automatically generated from the first and last names used in the Billing Address entered during checkout. If you require a custom username, then please request a PayPal invoice in advance.

For resellers who are unable to purchase online with PayPal or a credit card and encumber us with admin such as vendor forms, we reserve the right to charge an administration fee of £50.

How is the software delivered?

The software needs to be downloaded from our website, the licence key is delivered electronically by email.

What is the part number?

There is no part number or SKU.

What is the reseller price?

We do not offer discounted rates for resellers. The price is GBP at £99 per year, per user.

Where can I get company information?

On our contact page.

Where can I get licensing terms?

Licensing details can be found here.

Where can I get Form W-9 information?

Screaming Frog is a UK based company, so this is not applicable.

Can I get a quote in a currency other than GBP?

No, we only sell in GBP.

Setup
What hardware is recommended?
Where are my projects stored?

This varies depending on operating system, please replace USERNAME with the correct username for the operating system:

  • ​Windows: C:\Users\USERNAME\.ScreamingFrogLogfileAnalyser\projects\
  • ​Mac: /Users/USERNAME/.ScreamingFrogLogfileAnalyser/projects/
  • Linux: /home/USERNAME/.ScreamingFrogLogfileAnalyser/projects/

This can also be viewed and configured within the Log File Analyser, by navigating to 'Configuration > Workspace'. You can select 'Browse' and choose an alternative location for Log File Analyser projects.

Importing Logs
Why does the Log File Analyser not import my log file?

The most likely reason is the file you are importing is not a log file, or more specifically, an access log file. Please read our introduction to log files guide that gives examples of the supported log file formats so you can compare with your own.

Common formats imported that are not log files are:

  • CSV files
  • JSON files
  • Apache error logs

The Log File Analyser supports log files in W3C and Apache format. These logs are produced by the 3 most popular web servers: Apache, IIS and Nginx. There is no need to specify what format your logs are in, the Log File Analyser will auto detect this when you upload them.

If you have any problems importing logs, then please contact our support team.

What Log File formats are supported?

Please see our user guide section on importing log files that lists all our supported formats.

Please contact support if your log cannot be read.

Why can't I import CLF format Apache Log File?

The Apache Common Log Format (CLF) is usually the default format configured for many different web servers.

Unfortunately this format does not provide a User Agent field which is an integral part of the analysis provided by the Log File Analyser.

You will have to reconfigure or ask the administrator of the web server to reconfigure the log format string to provide the user agent field.

Here is the typical NCSA log format line required:

"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

What's the maximum supported Log File size?

The Log File Analyser uses a database to store the data. So the volume of logs that can be stored will be limited by the size of your hard drive. Performance will also be limited to how fast your disk is - an SSD will give you greater performance than a spinning disk.

The size of your site is not a relevant factor, it's how much data you have in your log files. A small but busy site could have a lot more log events than a large site that doesn't get much traffic.

We often import several Gigabytes worth of uncompressed log files without issue. The larger the time frame you request data for, the more events that have to be considered, and the slower the response will be.

How do you handle logs with multiple protocols and/or domains?

If you have an access log that contains entries for both http and https traffic, but the lines don't include the protocol, the Log File Analyser has no way of knowing which lines are for http and which are for https. The same is true if you have an access log that contains entries for multiple domains. Theses are both very much log file configuration issues. If no protocol and/or domain is specified, you'll be asked for it.

To include the protocol and domain for Apache style logs, the request line should be an absolute url. So rather than:

[01/Jan/2017:00:00:01 +0000] "GET /seo.html HTTP/1.1" 200 "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

The full protocol and domain will be on each line:

[01/Jan/2017:00:00:01 +0000] "GET http://example.com/seo.html HTTP/1.1" 200 "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

This way the Log File Analyser can clearly see the protocol and domain for each line.

For W3C logs, you'll need to include the cs-protocol (Advanced Logging needed for IIS) field along with either cs-host or x-host-header.

What does "No events were imported, no events match the selected user agents" mean?

The Log File Analyser discards all events (log lines) where the User Agent doesn't match those configured for the project. By default the Log File Analyser only imports events for a 'Bot' User Agents. If you receive this message it's likely you have imported a log that contains events for other User Agents (Chrome/Firefox etc) but not for any Bot User Agents (Googlebot/Bingbot etc). To disable all User Agent filtering create a new project and untick the "Filter User Agents" option when creating a new project (under the 'User Agents' tab). This option can only be set when creating a project. Log File Analyser user-agent configuration

Log File Analyser Results
Why can't I see all traffic?

By default the Log File Analyser only stores events from known search bots. This minimises the number of events imported and speeds up analysis. If you would like to see all events, untick the "Filter User Agents" option when creating a new project (under the 'User Agents' tab). This option can only be set when creating a project. Log File Analyser user-agent configuration

Why are the numbers different to Google Search Console?

  • Google Search Console (GSC) numbers are for all Google User Agents, the numbers shown in the Log File Analyser are on a per User Agent basis, and do not include all of Google's.
  • GSC figures are calculated using Pacific Daylight Time (PDT), however the Log File Analyser uses UTC by default. UTC is 7 hours ahead of PDT. You can adjust the UTC offset to match PDT by going to 'Project > Settings' and choosing UTC-07:00 from the dropdown.
  • 'Pages crawled per day' in GSC is all requests. If the same URL is requested 10 times, this is regarded as 10 pages.
  • Your log might be from a web server that sits behind a cache, so you're not seeing all the requests.
  • Your site might be load balanced across multiple servers and you've not imported logs from all the servers.

Why am I missing some fields from my results?

The Log File Analyser requires minimal information to import logs: Timestamp, URL, Response Code and User Agent. All other data is optional. If you are seeing "Average Response Time" of 0 then this will be because the log files imported did not include this information. To verify this you can look at the log files you imported to see what was included. If you are unfamiliar, please check our post An SEOs Guide To Log Files.

To add the missing data to your log files you will have to update you log configuration settings, which your web server administrator will be familiar with. Log formats supported by the Log File Analyser are detailed in each of the specific log file formats posts linked to from An SEOs Guide To Log Files.

Why am I seeing inconsistent response codes for a URL?

The Screaming Frog Log File Analyser is obviously analysing data over a period of time, directly from the server logs. So a URL might have been broken historically and subsequently been fixed, and hence why it has different or 'inconsistent' responses.

Another important thing to remember is, if you have uploaded a log file with relative URLs and had to provide the site URL on import, the non www. and www. version of URLs will be aggregated. So events to https://screamingfrog.co.uk and https://www.screamingfrog.co.uk would be aggregated to https://www.screamingfrog.co.uk if that's the address provided upon import. The non www. version might be set-up to 301 redirect to the www. version, and hence, half the responses redirect, while the other half provide 200 responses. This is why we generally recommend logs are configured with absolute URLs.

However, you may also find that a server is delivering inconsistent responses when under load.

Why am I seeing a different response code in a browser/the SEO Spider compared with the Log File Analyser

The Log File Analyser reads response codes from log files and displays them. If these are different to what you see by checking using a web browser or the SEO Spider there are a few things to check.

  • Some pages respond differently based on user agent, cookies etc. If you look through the history of response codes for the URL, by clicking the Events tab in the lower window pane, do the response codes vary?
  • Are the logs you've imported for the correct domain? If you had to provide a protocol and domain during import, there are a couple of potential issues. Perhaps the logs are for both the http and https versions of the site or maybe the logs are for multiple domains.

Windows
Why do I get a blank screen?

If the Log File Analyser user interface isn't rendering for you then the chances are you've run into this Java bug. On our experience this seems to be an issue with Intel HD 5xx series graphics cards. We've had less of these recently, so it might be a drive update will help resolve this issue.

If not, please close the SEO Spider, then open up the following file in a text editor:

C:\Program Files (x86)\Screaming Frog Log File Analyser\ScreamingFrogLogFileAnalyser.l4j.ini

then add the following under the -Xmx line:

-Dprism.order=sw

(You may have a permission issue here, so copying your desktop, editing, then copying back may be easier).

No when you start the Log File Analyser the user interface should render correctly.

Why do I get “error opening file for writing” when installing?

Try running the file as administrator by right clicking the installer and choosing “Run as administrator”. Alternatively log in to an administrator account. You may need to request assistance from your IT department depending on your company setup.

Why do I receive a 'protected your PC' message on installation in Windows 10?

When you install the Log File Analyser on Windows 10, you may receive a 'protected your PC' SmartScreen - Windows 10 SmartScreen To install, you have to click the 'More Info' link underneath and then 'Run Anyway' - smart screen more info link The reason for this is the certificate is new and it takes time for this message to disappear, despite being registered Windows developers.

Contact & Support
How do I send you debug logs?

You can get the debug logs from the Log File Analyser by going to Help->Debug->Save Logs. This will allow you to save a zip file containing the logs we need to help diagnose any issues.

If for some reason you are unable to start the Log File Analyser locate and attach the trace.txt file. This is found in a different location depending on your OS. Please substitute USERNAME for your own username.

Windows:
C:\Users\USERNAME\.ScreamingFrogLogfileAnalyser\trace.txt

OS X & Linux:
~/.ScreamingFrogLogfileAnalyser/trace.txt

How do I submit a bug / receive support?

Please follow the steps on the Log File Analyser support page so we can help you as quickly as possible. Please note, we only offer full support for premium users of the tool although we will generally try and fix any issues.

How do I provide feedback?

Feedback is welcome, please just follow the steps on the support page to submit feedback. Please note we will try to read all messages but might not be able to reply to all of them. We will update this FAQ as we receive additional questions and feedback.