Google-Extended user agent and robots.txt rules
Google-Extended is a robots.txt token, not a crawler. It controls whether pages Google already crawled may be used to train future Gemini models and to ground answers in Gemini Apps and Vertex AI.
User agent
No separate user agent string. Google documents the token Google-Extended only.
robots.txt rules for Google-Extended
Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read.
Block Google-Extended
Disallows every path for Google-Extended. That keeps your future pages out of what Google may use for training. It does not change search results.
User-agent: Google-Extended Disallow: /
Allow Google-Extended
Lets Google-Extended fetch every path, whatever “User-agent: *” says.
User-agent: Google-Extended Allow: /
Good to know
Google says it does not affect inclusion or ranking in Google Search, and it has no user agent string of its own.
Other Google crawlers
Googlebot
Googlebot is Google Search’s crawler. Google’s AI Overviews and AI Mode are Search features,…
OtherGoogle-CloudVertexBot
Google-CloudVertexBot runs crawls that site owners request to build Vertex AI Agents. Google…
User-triggeredGoogle-GeminiNotebook
Google-GeminiNotebook fetches the URLs that people add as sources to Gemini Notebook projects.…
User-triggeredGoogle-Agent
Google-Agent is used by agents on Google infrastructure that browse the web and act on a…
Google-Extended FAQ
What is Google-Extended?
Google-Extended is a robots.txt token, not a crawler. It controls whether pages Google already crawled may be used to train future Gemini models and to ground answers in Gemini Apps and Vertex AI.
How do I block Google-Extended?
Add “User-agent: Google-Extended” and “Disallow: /” to your robots.txt. That keeps your future pages out of what Google may use for training. It does not change search results.
Does Google-Extended respect robots.txt?
Google-Extended never crawls. It is a token you name in robots.txt to control how Google may use pages its crawler already fetched.
What is the Google-Extended user agent?
Google-Extended has no user agent string of its own; you write “Google-Extended” in robots.txt.
How can I verify Google-Extended requests?
There is nothing to verify: Google-Extended sends no requests.
Source: developers.google.com, checked Nov 17, 2026. AI Crawler Directory
Files are the start. Answers are the result.
Your buyers’ questions on eight AI engines, on your schedule, with history. Free for 7 days.