Google-Extended: What It Controls

Google-Extended is a robots.txt token that lets you manage whether content Google crawls from your site may be used to train future Gemini models and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI. Google says it doesn't affect your inclusion in Google Search and isn't a ranking signal (Google). It's not the control for AI Overviews or AI Mode.

Key takeaways

• Google-Extended covers Gemini training and Gemini/Vertex AI grounding.
• It doesn't affect Google Search, including AI Overviews and AI Mode.
• It's a control token, not a crawler; you won't see it in logs.
• Use snippet and indexing controls to manage Search AI features.

What exactly does Google-Extended control?

Google's crawler documentation lists two uses:

1. Training future generations of Gemini models that power Gemini Apps and the Vertex AI API for Gemini.
2. Grounding, defined as "providing content from the Google Search index to the model at prompt time to improve factuality and relevancy", in Gemini Apps and Grounding with Google Search on Vertex AI.

What doesn't it control?

How do you use it?

User-agent: Google-Extended
Disallow: /

Or more selectively, using Google's own example pattern:

User-agent: Google-Extended
Allow: /archive/1Q84
Disallow: /archive/

Because it's a token "used in a control capacity", Google applies the rules when deciding how content crawled by its normal user agents may be used (Google).

How do you control AI Overviews and AI Mode instead?

Google says AI features in Search use the same systems as Search and have no additional requirements; to appear, a page must be indexed and eligible to show a snippet (Google). To limit how content appears, Google points to standard controls:

These affect regular results too. There isn't a control that removes content only from AI features while leaving full snippets in classic results.

How do you decide whether to block it?

Ask:

• Is your content your product? Publishers and course creators often weigh training use more heavily.
• Do you want Gemini Apps to use your pages for grounded answers? Blocking Google-Extended also affects grounding, which may reduce how your current content is used in Gemini Apps answers.
• Are you consistent? Compare with your GPTBot and ClaudeBot decisions.

Illustrative. A software company wants maximum visibility, so it allows Google-Extended. A subscription publisher blocks it to keep content out of Gemini training and grounding, while staying in Search and AI Overviews through Googlebot.

What's changing?

Google says it intends to support the AIPREF shared machine-readable training preference signal, using user-agent: __usage_ai-training, in early 2027 (Google). If you manage training preferences, check this page again before then.

What if AI features still show content you limited?

Google's AI features page includes a troubleshooting note for this: make sure the preview control is correct and visible to Googlebot, and use the URL Inspection tool in Search Console to check what Google sees (Google). Changes only apply after Google recrawls the page, so allow time before rechecking.

How does this compare with other companies' controls?

One difference stands out: Google's grounding for Gemini Apps sits with the training control, while OpenAI and Anthropic separate search from training. If your policy is "allow anything that can cite us", allowing Google-Extended is consistent with that; if it's "no training anywhere", note that blocking Google-Extended also limits Gemini grounding.

How do you document the decision?

Add a short comment block to robots.txt and an internal note recording who decided, when and why. Illustrative:

# Google-Extended blocked 2026-10-30: content licensing policy (owner: legal)
User-agent: Google-Extended
Disallow: /

Comments in robots.txt are ignored by crawlers but help the next person who edits the file.

Who should be involved in the decision?

Google-Extended is partly a commercial and legal question, not only an SEO one. Bring in legal or licensing colleagues if your content is a product in itself, and keep marketing involved so that a training opt-out isn't confused with AI search visibility. Agree the decision once, document it and review it when Google's guidance changes.

Common mistakes

• Blocking Google-Extended expecting to leave AI Overviews.
• Avoiding it out of fear of losing rankings.
• Blocking Googlebot to stop AI features, which removes you from Search.
• Looking for it in logs.

Checklist

• [ ] Training and grounding policy agreed
• [ ] Google-Extended group written (if blocking)
• [ ] Search AI features handled via snippet/indexing controls, if needed
• [ ] Decision aligned with GPTBot and ClaudeBot policies
• [ ] Reminder set to review AIPREF support

Where to go next

• Google-Extended (glossary)
• List of AI crawlers and user agents

Sources

• Google, Google's common crawlers
• Google, AI features and your website
• Google, Block indexing with noindex