Importing Log Files

The Screaming Frog Log File Analyser is able to import log files from a variety of formats. The links below explain each format, provide examples logs to download and well as detail mandatory and optional fields.

• WC3
• Apache and NGINX
• Amazon Elastic Load Balancing
• HA Proxy
• JSON
• CSVs and TSVs

You can import single, or multiple log files and folders at a time and they can be compressed (zipped or gzipped). You do not need to specify the format of log files, the Log File Analyser will automatically recognise the format.

The Log File Analyser stores log event data in a local database, enabling it to analyse very large log files with millions of events. The larger the log file, the more time it will take to store the data on import and then generate ‘views’ in the tool (tabs, filtering and searching).

Drag & Drop Log Files

To import log files, drag and drop the log file(s) into the application in any tab, apart from the ‘Imported URL data’ tab. Alternatively, you can use the ‘Import > Log File’ button, or ‘Project > Import Log File’ option in the top level menu.

When you open a new project, the main window pane will show the following message where you can simply drag and drop the log file(s) –

Customise Log Fields & Debug

In the New Project configuration you can choose to ‘Show log fields configuration window on each import’ in the General tab.

While the LFA can automatically parse virtually all log formats, this is useful when log files are extremely customised, or a required field is missing.

You’re able to preview what’s in the log file and which log file components have been selected against which fields in the tool, and adjust if necessary.

This is an advanced feature, and in general is intended for more complex use cases.

Importing Additional Log Files Into A Project

You can continue to import additional data in the same way described above into an existing project, by dragging and dropping additional log files, which will add to the data in the project, rather than overwrite it.

If you make a mistake, or decide to remove some data, you can view (and delete) imports under ‘Project > Import History’.