Google-Extended, Googlebot and the Search generative AI control

#01Which control does what?

Google's Search generative AI control help page also says: "This control doesn't affect AI training; to limit training of the models used to generate responses in Search generative AI features, use Google-Extended."

#02How does the Search generative AI control work?

Timing: Google says a change generally takes a few days. Content is excluded within 1-2 days after the control goes live, and some may take longer because of caching.

Important details from the help page:

• Excluding doesn't remove you from classic results, and isn't used as a ranking signal.
• Content from other sites will still appear, "and it may appear similar to yours".
• Your content may still be used to power Search generally, such as understanding the language of queries.
• It doesn't override Merchant Center or Google Ads participation.

Google's generative AI optimisation guide now states that a site must be included under this control to be eligible for display in generative AI features. See Search Console's generative AI control for screenshots and the property hierarchy.

#03How does Google-Extended work?

Google-Extended has no separate crawler. Google crawls with its existing user agents and reads the Google-Extended robots.txt token as a usage control.

# Allow Search, opt out of Gemini training and Gemini Apps grounding
User-agent: Googlebot
Allow: /

User-agent: Google-Extended
Disallow: /

# Opt out only for a members-only research archive
User-agent: Google-Extended
Disallow: /research/members/

A shared AI training preference signal is discussed outside Google's crawler docs. Those docs, checked 5 November 2026, do not set an early-2027 date you can rely on. Wait until the crawler page names the signal before you change robots.txt for it.

#04Which setup fits which business?

Our view: for marketing sites, excluding from AI Overviews and AI Mode is rarely worth it. Your competitors' content "may appear similar to yours" in your place. Measure first with the Generative AI performance report, then decide.

#05How do snippet controls interact with AI features?

Google treats AI features as part of Search. To limit what's shown from your pages:

• nosnippet on a page: no text snippet, which makes it ineligible as a supporting link (it needs to be snippet-eligible).
• data-nosnippet on an element: keeps that element out of snippets and AI features while the rest of the page stays eligible. Useful for pricing small print or quotes you don't want shown out of context.
• max-snippet:[n]: caps snippet length.

If you set preview controls and still see content in AI features, Google suggests checking with URL Inspection that Googlebot sees the control, allowing time for recrawl, and requesting recrawl.

#06Worked example: Ledgerline

Ledgerline is a fictional accounts-payable software company. Details are illustrative.

A developer added "Disallow: /" for Google-Extended after reading a blog post, and someone else proposed excluding the site in Search Console "to protect content".

Walkthrough

1. The SEO lead pulled the Generative AI performance report: 14,000 impressions a month (illustrative) for key product pages.
2. She mapped each proposal to Google's documentation, using the table above.
3. Leadership chose to stay visible in AI Search and Gemini, but keep members-only research out of Gemini training and grounding.
4. robots.txt was updated, and URL Inspection confirmed Googlebot could still fetch key pages.
5. The decision and its reasons were written into the site governance doc.

#07Common mistakes

• Blocking Google-Extended to "leave AI Overviews". It doesn't work.
• Blocking Googlebot and losing Search entirely.
• Excluding a parent property without realising child properties inherit it.
• nosnippet on pages you want cited.
• Expecting instant effects. Allow days for the control, and longer for recrawls.

#08Try it yourself

Open your robots.txt and Search Console > Settings > Search generative AI. Write down the current state of each control in a four-row table (Googlebot, Search generative AI control, Google-Extended, snippet controls). Then answer the three questions from the Pro move, and note any control that doesn't match your answers.

#09Quick quiz

1. Does Google-Extended affect AI Mode?

Answer: No. It doesn't affect Search inclusion. Use the Search generative AI control.

2. What are the three Search generative AI control options?

Answer: Include, Exclude and Inherit.

3. What does data-nosnippet do?

Answer: It keeps a specific element out of snippets and AI features.

#10What should you check before the next lesson?

• Every Google control is documented with an owner.
• Marketing pages are crawlable and snippet-eligible.
• The decision on Google-Extended is written down.

Previous: How Gemini, AI Mode and AI Overviews choose sources. Next: Structured data, entities and the Knowledge Graph for Gemini. Or return to the Gemini for Marketers track. Related: Configure robots.txt for AI crawlers, Blocking or allowing AI crawlers, PerplexityBot, robots.txt and WAF rules.

#11Common questions

Does blocking Google-Extended remove my site from AI Overviews? No. Google-Extended does not affect inclusion in Google Search. To exclude your site from AI Overviews, AI Mode and Discover's generative AI features, use the Search generative AI control in Search Console.

What does Google-Extended control? Whether content Google crawls from your site may be used to train future Gemini models, and for grounding in Gemini Apps and Grounding with Google Search on Google Cloud. It has no separate crawler user agent.

How long does the Search generative AI control take to apply? Google says a change generally takes a few days. Content is excluded within 1-2 days after the control goes live, though some content can take longer because of caching.