JavaScript: Nofollow Only in Original HTML

Pages that contain a meta robots nofollow in the raw HTML, and not in the rendered HTML.

This means any hyperlinks in the raw HTML pre to JavaScript execution will not be followed.

How to Analyse in the SEO Spider

Enable JavaScript rendering mode via ‘Config > Spider > Rendering’ and select ‘JavaScript’ to crawl JavaScript websites.

View URLs with this issue in the ‘JavaScript’ tab and ‘Nofollow Only in Original HTML’ filter and export all URLs using the ‘Export’ button.

The ‘HTML Meta Robots’ column shows the robots directives found in the original HTML before JavaScript.

The ‘Rendered HTML Meta Robots’ column shows robots directives found in the rendered HTML after JavaScript has been processed.

Read our tutorial on ‘How To Crawl JavaScript Websites‘.

What Triggers This Issue

Pages that contain a meta robots ‘nofollow’ directive in the raw HTML, and not in the rendered HTML. This means any hyperlinks in the raw HTML pre to JavaScript execution will not be followed.

For example:

https://www.screamingfrog.co.uk/

Has the meta tag:

<meta name="robots" content="nofollow">

Which is then removed after JavaScript is executed.

How To Fix

Carefully review pages with nofollow in the raw HTML are expected not to be followed. Remove the ‘nofollow’ if links should be followed, crawled and indexed.