- Response Codes
- Internal No Response
- Internal Client Error (4XX)
- Internal Server Error (5XX)
- Internal Redirect Loop
- Internal Blocked by Robots.txt
- Internal Blocked Resource
- Internal Redirect Chain
- External Blocked Resource
- Internal Redirection (3XX)
- Internal Redirection (Meta Refresh)
- Internal Redirection (JavaScript)
- External No Response
- External Client Error (4XX)
- External Server Error (5XX)
- Security
- HTTP URLs
- Mixed Content
- Form URL Insecure
- Form On HTTP URL
- Missing HSTS Header
- Unsafe Cross Origin Links
- Protocol-Relative Resource Links
- Missing Content-Security-Policy Header
- Missing X-Content-Type-Options Header
- Missing X-Frames-Options Header
- Missing Secure Referrer-Policy Header
- Bad Content Type
- Hreflang
- Non-200 Hreflang URLs
- Missing Return Links
- Inconsistent Language & Region Confirmation Links
- Non-Canonical Return Links
- Noindex Returns Links
- Incorrect Language & Region Codes
- Multiple Entries
- Not Using Canonical
- Outside <head>
- Unlinked Hreflang URLs
- Missing Self Reference
- Missing X-Default
- JavaScript
- Noindex Only in Original HTML
- Nofollow Only in Original HTML
- Canonical Mismatch
- Uses Old AJAX Crawling Scheme URLs
- Uses Old AJAX Crawling Scheme Meta Fragment Tag
- Pages with Blocked Resources
- Contains JavaScript Links
- Contains JavaScript Content
- Page Title Only in Rendered HTML
- Page Title Updated by JavaScript
- Meta Description Only in Rendered HTML
- Meta Description Updated by JavaScript
- H1 Only in Rendered HTML
- H1 Updated by JavaScript
- Canonical Only in Rendered HTML
- Pages With JavaScript Errors
- Links
- Outlinks To Localhost
- Pages Without Internal Outlinks
- Non-Indexable Page Inlinks Only
- Internal Nofollow Outlinks
- Pages With High External Outlinks
- Pages With High Internal Outlinks
- Follow & Nofollow Internal Inlinks To Page
- Internal Nofollow Inlinks Only
- Pages With High Crawl Depth
- Internal Outlinks With No Anchor Text
- Non-Descriptive Anchor Text In Internal Outlinks
- AMP
- Non-200 Response
- Missing Non-AMP Return Link
- Missing Canonical to Non-AMP
- Non-Indexable Canonical
- Missing <html amp> Tag
- Missing/Invalid Doctype HTML Tag
- Missing Head Tag
- Missing Body Tag
- Missing Canonical
- Missing/Invalid Meta Charset Tag
- Missing/Invalid Meta Viewport Tag
- Missing/Invalid AMP Script
- Missing/Invalid AMP Boilerplate
- Contains Disallowed HTML
- Other Validation Errors
- Indexable
- Structured Data
- Validation Errors
- Missing
- Validation Warnings
- PageSpeed
- Eliminate Render-Blocking Resources
- Properly Size Images
- Defer Offscreen Images
- Minify CSS
- Minify JavaScript
- Reduce Unused CSS
- Reduce Unused JavaScript
- Efficiently Encode Images
- Serve Images in Next-Gen Formats
- Enable Text Compression
- Preconnect to Required Origin
- Reduce Server Response Times (TTFB)
- Preload Key Requests
- Reduce JavaScript Execution Time
- Serve Static Assets With An Efficient Cache Policy
- Minimize Main-Thread Work
- Image Elements Do Not Have Explicit Width & Height
- Avoid Large Layout Shifts
- Avoid Serving Legacy JavaScript to Modern Browsers
- Avoid Multiple Page Redirects
- Use Video Format for Animated Images
- Avoid Excessive DOM Size
- Ensure Text Remains Visible During Webfont Load
Internal Search
URLs that might be part of the websites internal search function.
Google and other search engines recommend blocking internal search pages from being crawled to limit sometimes duplicate and low quality pages from being crawled and indexed.
How to Analyse in the SEO Spider
View URLs with this issue in the URL tab and ‘Internal Search’ filter, and export all URLs using the ‘Export’ button. Use the lower ‘Inlinks’ tab to view pages that link to them.
What Triggers This Issue
This issue is triggered when a URL appears to be part of the website’s internal search function. For example:
https://www.screamingfrog.co.uk/search?q=example
How To Fix
Most internal site searches are built for users, rather than search engines who may needlessly crawl and index them.
As best practise most search sections of a site should not be linked to internally, and should be disallowed in robots.txt.
Popular search pages could be built into real pages to help improve organic traffic.
Further Reading
- URL Structure - Learn SEO - From Screaming Frog
- URL Structure Best Practices For Google - From Google