Gemini

Configuration > API Access > AI > Gemini

To connect to the Gemini API, you need an API key. You can create a key in Google AI Studio.

Click ‘Create API Key’, select a project and set up an API key. Gemini is only available in select regions, and there are free and paid account types with different rate limits.

Please note: The Gemini free tier is extremely limited now. The most economical chat model is restricted to just 5 requests per minute (RPM), while their default embeddings model is limited to 100 RPM, but in practice appears to error over 20 RPM (doh, Google!).

When you have an API key, copy and paste it into the ‘API Key’ field in the Gemini Account Information tab.

Click ‘Connect’ when you wish to use the set up for a crawl.

Navigate to the ‘Prompt Configuration’ tab, to set up up to 100 prompts against crawl data.

Select the category of model (Gemini or Embeddings), the Gemini model used (for example, ‘gemini-2.5-flash’), content type and data to be used for the prompt, such as body text, HTML, or a custom extraction, as well as write your custom prompt.

Please note: To use ‘Page Text’ or ‘HTML’ for the prompt, you will need to enable ‘Store HTML‘ via ‘Config > Spider > Extraction’.

To test a prompt, use the ‘play’ icon to the right of the prompt field.

In the Gemini prompt tester, input the URL to test and click the ‘Test’ button to display both the extraction, and response.

In the Advanced tab, you can adjust the maximum requests per minute for different models.

The two most common models are for chat and embeddings. If you’re a paid user, check your rate limits in AI Studio, and increase the RPM to get results faster.

At the time of writing, Gemini paid tier 1 rate limits are 1k RPM for chat, while embeddings are 3k RPM.

If you’re receiving ‘Error: Too Many Requests, You exceeded your current quota, please check your plan and billing details.”, then you will need to reduce the requests per minute.

Data from the prompt will appear in the AI tab (and Internal tab, against your usual crawl data).

The ‘Add from Library’ function includes half a dozen prompts for inspiration.

You’re able to add and customise your own using the ‘Add from Library’ button, clicking ‘User’ and the ‘+’ button.

Custom prompts can be shared using the export and import buttons at the top.

Read our full tutorial on How To Crawl With AI Prompts.