Google-Extended

What it is

Google-Extended is a robots.txt token Google runs to give you a say in how your content is used for its AI products. It is not a separate crawler with its own IP range and it does not fetch anything new. Googlebot still does the crawling. Google-Extended is a control layer on top: a User-agent: Google-Extended block with Disallow: / tells Google not to use your already-crawled pages to train its Gemini models or to ground Gemini's answers. Robots.txt does apply here. Allowing or blocking it does not affect your normal Search rankings, because Googlebot indexes your site regardless.

Why it matters

The trap is assuming Google-Extended controls AI Overviews. It does not. AI Overviews and AI Mode draw from the regular Search index, so blocking Google-Extended does not remove you from them. Google says this plainly in its AI features optimization guidance: the work for AI features is ordinary good SEO, governed by the standard index, not by a special opt-out.

So the real decision is narrow. Block Google-Extended if you want to keep your content out of Gemini training and grounding. Leave it open if you want to be eligible for those surfaces. For how this compares to the other crawlers, see AI crawlers explained.