Tutorials
For the SEO Spider
Site Architecture & Crawl Visualisations Guide
Visualise site architecture and internal linking to help communicate ideas and reveal underlying patterns that are harder to uncover in data, and spreadsheets.
How To Crawl JavaScript Websites
Render web pages using the integrated Chromium WRS to crawl dynamic, JavaScript rich websites and frameworks, such as Angular, React and Vue.js.
How To Crawl A Staging Website
Find out how to crawl a staging or development website, considering robots.txt, authentication, and the SEO Spider configuration.
How To Automate Crawl Reports In Looker Studio
Learn how to set-up fully automated Google Looker Studio crawl reports to monitor site health, detect issues, & track performance.
How To Audit Core Web Vitals
Analyse which pages pass or fail Google's Core Web Vitals assessment using field data from CrUX to improve page speed.
How To Test & Validate Structured Data
Audit and validate structured data against Schema.org specifications, and Google’s search feature requirements to enable special search result features.
How To Find Missing Image Alt Text & Attributes
Find pages that are missing image alt text and alt attributes, preview the images and bulk export the source pages and images to be corrected.
Web Scraping & Custom Extraction
Scrape any data from the HTML of a page using CSS Path, XPath and regex to enhance a crawl, such as author name, comments, shares or more.
How To Audit Redirects In A Site Migration Using The SEO Spider
Learn how to crawl a list of URLs, follow redirect chains, identify the final target & identify errors & non-indexable pages.
How To Audit Canonicals
A step-by-step tutorial on how to audit canonical link elements found in the HTML and HTTP Headers for common errors and issues.
How To Audit & Validate Accelerated Mobile Pages (AMP)
This tutorial walks you through how to audit Accelerated Mobile Pages (AMP). The SEO Spider uses the official AMP validator to allow bulk URL validation.
How To Audit rel="next" and rel="prev" Pagination Attributes
While Google no longer use rel=”next” and rel=”prev” pagination attributes, Bing use them as a hint. Learn how to audit your pagination set-up.
How To Audit XML Sitemaps
XML Sitemaps should be up to date, error free, and include indexable, canonical URLs only. Learn how to audit them for common issues.
How To Bulk Check Redirects
Check redirects by crawling your site, or by uploading a list of URLs in bulk. Discover if they are 301s or 302s, loops and chains.
How To Find Orphan Pages
Connect to Google Analytics and Search Console, integrate XML Sitemaps and discover orphan pages, that are not linked to internally from the website.
Robots.txt Testing In The SEO Spider
View URLs blocked by robots.txt, the disallow lines & use the custom robots.txt to check & validate a site's robots.txt thoroughly, and at scale.
XML Sitemap Generator
Learn how to generate an XML Sitemap (and image sitemap), including the URLs, last modified, priority and change frequency you want.
How To Use The SEO Spider In A Site Migration
Learn how to use the SEO Spider's crawling and features in a site migration to work more efficiently, and improve chances of success.
How To Bypass Geo IP Redirection In A Crawl
Learn how to set your preferred country using web form authentication to crawl a site when it uses automatic GEO IP redirection.
How To Perform A Cookie Audit
Perform a cookie audit for compliance to identify all cookies that are being automatically set without explicit user consent at scale.
How To Analyse Link Position
Analyse the position of links found in a crawl to identify and fix links faster, and improve internal linking for users and search engines.
How To Perform A Parity Audit
Learn how to perform a JavaScript and mobile vs desktop parity audit to uncover differences and potential SEO issues.
How To Automate The URL Inspection API
Collect URL Inspection API data in bulk, work with the 2k limit, & automate URL Inspection data reporting to monitor indexing.
How To Debug Invalid HTML Elements In The Head
Identify invalid HTML Elements in the <head>, view metadata affected, and debug what's actually causing the issue.
An SEOs Guide To Crawling HSTS
HSTS declares to a client that it should only be accessed via HTTPS & can show as a 307 redirect. Find out why & how to see the real status code.
