Cookies

Google crawls the web stateless without cookies, but will accept them for the duration of a page load. Some websites can only be viewed when cookies are accepted, and fail when accepting them is disabled.

By default the SEO Spider will accept cookies for a ‘session only’. This means they are accepted for the page load, where they are then cleared and not used for additional requests in the same way as Googlebot.

Cookie storage can be adjusted under ‘Configuration > Spider > Advanced’ from the default ‘Session Only’, to either ‘Persistent’ or, ‘Do Not Store’.