Directives

Dan Sharp

Posted 28 October, 2019 by in

Directives

Configuration > Spider > Extraction > Directives

The following directives are configurable to be stored in the SEO Spider.

  • Meta Robots
  • X-Robots-Tag

Disabling any of the above options from being extracted will mean they will not appear within the SEO Spider interface in respective tabs, columns or filters.

A small amount of memory will be saved from not storing the data.

Dan Sharp is founder & Director of Screaming Frog. He has developed search strategies for a variety of clients from international brands to small and medium-sized businesses and designed and managed the build of the innovative SEO Spider software.

Like us on Facebook Connect with us on LinkedIn Follow us on Twitter Subscribe to our channel us YouTube View our RSS feed

Join the mailing list for updates, tips & giveaways

How we use the data in this form

Get in touch

For an in-depth proposal on our services, complete our contact form to request a proposal.

Contact us

We'll get back to you asap.

Comments are closed.

Back to top