Course lesson · Beginner
SEO Basics You Still Need
The SEO basics you still need are the ones that let machines find, understand and trust a page: it must be crawlable, indexable, clearly titled and structured, linked from other pages and genuinely helpful. Google says pages must be indexed and eligible for a snippet to appear in its AI features (Google), so these basics are the entry ticket for AI search too. This lesson gives you a practical checklist for any page.
In this lesson
- Crawlability and indexability
- Titles, headings and structure
- Links: internal and external
- Helpful content
- Exercise: a one-page audit
Can search engines and AI crawlers reach the page?
A page that can't be fetched can't be used. Check:
| Check | How | Common problem |
|---|---|---|
| robots.txt | Visit yoursite.com/robots.txt | A Disallow rule blocking important folders |
| Server response | Load the page; check it returns normally | Errors, login walls, aggressive bot protection |
| AI crawler rules | Look for groups such as OAI-SearchBot or PerplexityBot | Blocking search crawlers by mistake |
robots.txt controls crawling, not indexing (Google). AI companies document their own crawlers; for example, OpenAI says sites that opt out of OAI-SearchBot won't be shown in ChatGPT search answers (OpenAI).
Can the page be indexed?
A crawled page still might not be indexed. Common reasons:
- A
noindextag or header tells search engines to leave it out (Google). - It duplicates another page, so the search engine picks one version.
- It has very little unique content.
- Content only appears after complex JavaScript runs and isn't rendered properly (Google).
In Google Search Console, the URL Inspection tool shows whether a specific page is indexed.
Is the page clearly titled and structured?
| Element | Good practice |
|---|---|
| Title | Unique, descriptive, matches the page's main topic (Google) |
| Main heading (H1) | Says what the page is about in plain words |
| Subheadings (H2/H3) | Break the page into logical sections; questions work well |
| Opening | Answers the main question early |
| Lists and tables | Present steps and comparisons clearly |
Clear structure helps people skim, and it helps systems identify which part of a page answers which question.
How do links help?
Internal links connect your own pages. They help crawlers discover pages and show which pages matter. A new guide with no links pointing to it may be found slowly or not at all.
External links and mentions from other sites help search engines discover your pages and are among the signals of reputation. For AI answers, what other sites say about you also shapes descriptions.
Practical steps:
- Link new pages from relevant existing pages and navigation.
- Use descriptive link text ("pricing for agencies", not "click here").
- Keep an XML sitemap up to date (Google).
What does "helpful content" mean in practice?
Google's guidance asks whether content is created primarily for people, demonstrates expertise and leaves readers satisfied (Google). In practice:
- Answer the question the page targets, fully and accurately.
- Show first-hand experience or evidence where relevant.
- Keep facts current, with dates where they change.
- Avoid filler written only to hit a word count.
What about structured data?
Structured data is code that labels information, such as products, reviews or organisations, in a format search engines understand (Google). It can enable rich results. Google says there's no special schema required for AI features (Google), so treat it as helpful for clarity, not as an AI shortcut.
A quick example
Illustrative. A company's best guide isn't appearing in AI answers. An audit finds three issues: its CMS added noindex to all pages in the /resources/ folder, the page title is just "Resource 14", and no other page links to it. None of these is an "AI problem", but fixing them makes the page eligible to be found, understood and cited.
Exercise
Do a one-page on-page audit of one URL. Check and note: title, H1, subheadings, whether the opening answers the main question, internal links pointing to it, indexability (noindex? appears in Search Console?), and whether robots.txt allows the main search and AI search crawlers.
Quiz
- Does robots.txt control indexing?
a) Yes, always b) No, it controls crawling c) Only for images Answer: b
- What does a noindex tag do?
a) Speeds up the page b) Tells search engines not to include the page in results c) Adds structured data Answer: b
- Why do internal links matter?
a) They help crawlers discover pages and show importance b) They replace content c) They're only for ads Answer: a
- According to Google, is special schema required for AI features?
a) Yes b) No c) Only for products Answer: b
- Which is a sign of helpful content?
a) Repeated keywords b) It answers the question fully and accurately for people c) It's over 3,000 words Answer: b
Next: Lesson 4: Content that answers questions
Sources
- Google Search Central, AI features and your website
- Google Search Central, Introduction to robots.txt
- Google Search Central, Block indexing with noindex
- Google Search Central, JavaScript SEO basics
- Google Search Central, Influencing title links
- Google Search Central, Sitemaps overview
- Google Search Central, Creating helpful content
- Google Search Central, Intro to structured data
- OpenAI, Overview of OpenAI crawlers
Questions
Is SEO dead because of AI?
No. AI search features rely on crawling and indexing, so SEO fundamentals are still the base layer.
Which SEO basics matter most for AI visibility?
Being crawlable and indexable, clear page structure and accurate, helpful content.
Do I need structured data?
It helps search engines understand some content and enables some rich results, but Google says no special schema is needed for AI features.