Custom search

Custom search

Configuration > Custom > Search

The SEO Spider allows you to find anything you want in the source code of a website. The custom search feature will check the HTML (page text, or specific element you choose to search in) of every page you crawl.

By default custom search checks the raw HTML source code of a website, which might not be the text that is rendered in your browser. You can switch to JavaScript rendering mode to search the rendered HTML.

You’re able to configure up to 100 search filters in the custom search configuration, which allow you to input your text or regex and find pages that either ‘contain’ or ‘does not contain’ your chosen input.

This can be found under ‘Config > Custom > Search’.

Simply click ‘Add’ (in the bottom right) to include a filter in the configuration.

From left to right, you can name the search filter, select ‘contains’ or ‘does not contain’, choose ‘text’ or ‘regex’, input your search query – and choose where the search is performed (HTML, page text, an element, or XPath and more).

For example, you may wish to choose ‘contains’ for pages like ‘Out of stock’ as you wish to find any pages which have this on them. When searching for something like Google Analytics code, it would make more sense to choose the ‘does not contain’ filter to find pages that do not include the code (rather than just list all those that do!).

The pages that either ‘contain’ or ‘does not contain’ the entered data can be viewed within the ‘Custom Search’ tab.

The ‘contains’ filter will show the number of occurrences of the search, while a ‘does not contain’ search will either return ‘Contains’ or ‘Does Not Contain’.

In this search, there are 2 pages with ‘Out of stock’ text, each containing the word just once – while the GTM code was not found on any of the 10 pages.

The SEO Spider uses the Java regex library, as described here. To ‘scrape’ or extract data, please use the custom extraction feature.

You are able to use regular expressions in custom search to find exact words. For example –

\bexample\b

Would match a particular word (‘example’ in this case), as \b matches word boundaries.

Please see our tutorial on ‘How to Use Custom Search‘ for more advanced scenarios, such as case sensitivity, finding exact & multiple words, combining searches, searching in specific elements and for multi-line snippets of code.