Guide · Beginner
SEO Basics for the AI Era: How Search Still Works
Search still works in three stages: crawling, indexing and serving results. AI search features sit on top of this. Google says that to appear in AI Overviews or AI Mode, a page must be indexed and eligible to show a snippet, and no special markup is needed. Getting the SEO basics right is therefore the entry ticket to AI visibility.
How does Google Search work?
Google describes three stages (Google):
| Stage | What happens | What you control |
|---|---|---|
| Crawling | Googlebot discovers and downloads pages | Links, sitemaps, robots.txt, server health |
| Indexing | Google analyses the content and stores it | Unique, readable content, canonical tags, no accidental noindex |
| Serving | Google picks the most relevant, helpful results | Content quality, relevance, page experience |
Not every page is crawled, and not every crawled page is indexed. Google says it doesn't guarantee crawling, indexing or ranking, even if you follow its guidelines.
How do AI features use this?
Google's guidance on AI features says the best practices for SEO remain relevant for AI Overviews and AI Mode. There are no additional requirements: a page must be indexed and eligible to appear with a snippet (Google). AI Mode and AI Overviews may use a "query fan-out" technique, running multiple related searches to find supporting pages (Google).
Other engines work similarly. ChatGPT search, Perplexity and Copilot use their own crawlers or search indexes, such as OAI-SearchBot (OpenAI), PerplexityBot (Perplexity) and Bingbot (Bing). If those crawlers can't reach your page, it can't be cited.
What are the SEO basics, step by step?
1. Make pages crawlable
- Link every important page from other pages on your site.
- Submit an XML sitemap (Google).
- Don't block important pages in robots.txt (Google).
- Check that content renders without relying on blocked JavaScript (Google).
2. Make pages indexable
- Avoid accidental
noindextags (Google). - Consolidate duplicates with canonical tags (Google).
- Give each page a clear, unique purpose.
3. Make pages understandable
- Write descriptive titles; Google generates title links from them (Google).
- Use headings that reflect the content structure.
- Add structured data where relevant (Google).
4. Make pages helpful
Google's guidance on helpful content asks whether content is original, complete, trustworthy and written for people first (Google). This also matters for AI answers: engines prefer clear, specific and trustworthy passages to quote.
Can you pay to rank or be included?
No. Google says it doesn't accept payment to crawl a site more often or rank it higher (Google). The same applies to AI Overviews: meeting the requirements makes a page eligible but doesn't guarantee it will be shown (Google). Be wary of anyone selling guaranteed placement in AI answers.
What has changed in the AI era?
| Classic SEO focus | Added AI-era focus |
|---|---|
| Rank for keywords | Be quoted or recommended for questions |
| Title and meta for click | Answer-first passages that stand alone |
| Backlinks | Mentions and citations on trusted sources |
| One search engine | Several engines and assistants |
| Traffic as the main metric | Mention, citation and recommendation rates as well |
What stays the same?
- Clear technical foundations
- Content that answers real questions well
- Signals of trust: authorship, sources, accurate facts
- Avoiding spam tactics, which Google's spam policies prohibit (Google)
Is AI-generated content allowed?
Google says that using automation, including AI, to generate content primarily to manipulate rankings violates its spam policies. But it rewards helpful content however it's produced (Google). The quality bar is the same.
A quick example
Illustrative. A startup publishes an excellent comparison guide but deploys it with a leftover noindex tag from staging. It never appears in Google, AI Overviews or AI Mode. After the tag is removed and the URL is submitted in a sitemap, the page is indexed within weeks and begins to appear as a supporting link.
What mistakes do beginners make?
- Blocking crawlers by accident in robots.txt or with noindex
- Publishing thin, near-duplicate pages
- Hiding the main content behind scripts or logins
- Ignoring Search Console warnings (Google)
- Thinking AI search needs special markup instead of good basics
Beginner checklist
- [ ] Important pages linked internally and in a sitemap
- [ ] robots.txt allows search and AI search crawlers you want
- [ ] No accidental noindex
- [ ] Unique, descriptive titles and headings
- [ ] Helpful, original content with clear answers
- [ ] Search Console set up and checked monthly
Related
Sources
- Google Search Central, How Search works
- Google Search Central, AI features and your website
- Google Search Central, SEO Starter Guide
- Google Search Central, Creating helpful, reliable, people-first content
- Google Search Central, Spam policies
- Google Search Central, Google Search's guidance about AI-generated content
- OpenAI, Overview of OpenAI crawlers
- Perplexity, Perplexity crawlers
Questions
Does SEO still matter with AI search?
Yes. AI search features built on web search, such as Google AI Overviews, rely on pages being crawlable and indexed. Google says the same SEO best practices apply.
What are the three stages of Google Search?
Crawling, indexing and serving search results, as described in Google's How Search Works documentation.
Do I need special markup for AI Overviews?
Google says no special optimisation or markup is required to appear in AI features; pages must be indexed and eligible to show a snippet.