Technical GEO
Technical GEO for AI search: AI crawlers and robots.txt, llms.txt, schema markup, rendering, indexing and how to keep pages retrievable by answer engines.
Technical GEO makes sure AI crawlers and search engines can reach, read and understand your pages: crawler access, rendering, structured data, sitemaps and files like llms.txt. Without it, nothing else in GEO can work.
How AI search builds an answer: deciding to search, retrieval, passage ranking and synthesis, with MentionX citation data and what you control.
Read it first →Start with these three
What llms.txt is, what goes in it, which AI systems read it, and whether it's worth adding, with MentionX citation data on .md and llms.txt URLs.
Guide · 11 min02How to Create an llms.txt File (With Example)Step-by-step guide to creating an llms.txt file: choose pages, write the summary, format links, publish, test, and maintain it, with MentionX data.
Tutorial · 11 min03llms.txt vs robots.txtllms.txt vs robots.txt: what each file does, how they differ, which setup fits your site, and how to keep them consistent, with MentionX data.
Guide · 11 minAll 31 guides
In reading order- What Is llms.txt? Complete GuideGuidePractitioner11 min
- How to Create an llms.txt File (With Example)TutorialPractitioner11 min
- llms.txt vs robots.txtGuidePractitioner11 min
- GPTBot User Agent: Allow or Block?GuidePractitioner11 min
- How to Configure robots.txt for AI CrawlersTutorialPractitioner10 min
- Schema Markup for AI SearchGuidePractitioner11 min
- List of AI Crawlers and User Agents (2026)GuidePractitioner12 min
- OAI-SearchBot vs GPTBot vs ChatGPT-UserGuidePractitioner5 min
- PerplexityBot and Perplexity-User ExplainedGuidePractitioner5 min
- Google-Extended: What It ControlsGuidePractitioner4 min
- ClaudeBot and Claude-SearchBotGuidePractitioner4 min
- Reading Server Logs for AI Bot VisitsTutorialPractitioner10 min
- Cloudflare AI Bot Blocking: Check You're Not InvisibleTutorialPractitioner10 min
- Organization Schema for Brand EntitiesTutorialPractitioner11 min
- FAQPage Schema: Still Worth It?GuidePractitioner10 min
- HowTo Schema After Google's ChangesGuidePractitioner10 min
- Article and Author Schema for ExpertiseTutorialPractitioner10 min
- SoftwareApplication Schema for SaaSTutorialPractitioner10 min
- Review and AggregateRating SchemaTutorialPractitioner10 min
- DefinedTerm Schema for GlossariesTutorialPractitioner10 min
- Dataset Schema for Research PagesTutorialPractitioner11 min
- JavaScript Rendering and AI CrawlersGuidePractitioner11 min
- Page Speed and AI RetrievalGuidePractitioner10 min
- Canonicals, Duplicates and AI AnswersGuidePractitioner10 min
- XML Sitemaps for AI DiscoveryTutorialPractitioner10 min
- Bing Webmaster Tools for AI VisibilityTutorialPractitioner10 min
- Markdown Versions of Pages for LLMsGuidePractitioner10 min
- Paywalls, Logins and AI AccessGuidePractitioner10 min
- Structured Data Testing for AITutorialPractitioner11 min
- Wikidata for Brands: How and WhenTutorialPractitioner10 min
- MCP Servers and Agent-Readiness for WebsitesGuidePractitioner10 min
Related topics
Common questions
From the guides in this topicIs llms.txt an official standard?
+
No. It's a community proposal published at llmstxt.org in 2024. It isn't an IETF or W3C standard, and no major AI company has announced it as a requirement for inclusion in answers. From: What Is llms.txt? Complete Guide
Where does llms.txt go?
+
At the root of your domain, so it loads at https://yourdomain.com/llms.txt, served as plain text. From: How to Create an llms.txt File (With Example)
Can llms.txt replace robots.txt?
+
No. robots.txt is the standard way to tell crawlers what they may fetch. llms.txt has no access-control function at all. From: llms.txt vs robots.txt