Meta refresh
Dan Sharp
Posted 24 October, 2019 by Dan Sharp in
Meta refresh
Configuration > Spider > Crawl > Meta Refresh
By default the SEO Spider will store and crawl URLs contained within a meta refresh.
<meta http-equiv="refresh" content="4; URL='www.screamingfrog.co.uk/meta-refresh-url'"/>
Unticking the ‘store’ configuration will mean meta refresh details will not be stored and will not appear within the SEO Spider.
Unticking the ‘crawl’ configuration will mean URLs discovered within a meta refresh will not be crawled.