Markdown Versions of Pages for LLMs

Publishing Markdown versions of pages gives AI agents a clean, low-token copy of your content. There are two main approaches: separate `.md` URLs, as proposed by llms.txt, or serving Markdown on the same URL when a client sends `Accept: text/markdown` ([llms.txt](https://llmstxt.org/), [Cloudflare](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/)). Neither is required for Google's AI features ([Google](https://developers.google.com/search/docs/appearance/ai-features)).

Facts are from llms.txt, Cloudflare, MDN and Google, accessed 30 Oct 2026. Points marked "Our view" are interpretation.

On this page

• What are the two approaches?
• How does Cloudflare's Markdown for Agents work?
• What are the risks?
• When does it make sense?
• How do you implement .md URLs well?
• How will you know it's working?
• How does this relate to llms.txt?
• Exercise
• Checklist
• From MentionX data: which pages agents are likely to want
• Worked example: Ledgerline adds Markdown to its docs
• Why would an agent want Markdown?
• What should a Markdown version include?
• How do you signal that a Markdown version exists?
• Should Markdown files be indexed by search engines?
• What are the costs?
• How do you test Markdown output?
• Which teams benefit most?
• How do you measure use?
• How do you handle pages that change often?
• What about pages behind a login?
• Should you tell users about Markdown versions?
• Common questions
• Related
• Sources

Checked on 31 October 2026. "Our view" marks MentionX opinion. Ledgerline is a fictional company used for illustration.

What are the two approaches?

The llms.txt proposal also recommends link relations so clients can find them: rel="alternate" type="text/markdown" pointing to the Markdown version, and rel="describedby" pointing to the covering llms.txt file, either as HTML <link> elements or an HTTP Link header (llms.txt).

<link rel="alternate" type="text/markdown" href="/learn/guide.md">

How does Cloudflare's Markdown for Agents work?

From Cloudflare's documentation (Cloudflare):

• On zones with it enabled, a request with Accept: text/markdown makes Cloudflare fetch the HTML from your origin and convert it to Markdown on the fly.
• The response has Content-Type: text/markdown; charset=utf-8 and Vary includes Accept, so caches keep HTML and Markdown variants separate.
• Origin headers such as HSTS, CSP, Set-Cookie, CORS and caching headers are preserved. ETag and Last-Modified are dropped, so conditional requests can't be honored.
• Headers, footers, navigation, scripts and styles are stripped; metadata from <meta> tags becomes YAML frontmatter.
• Content signals: if your origin doesn't send a content-signal header, Cloudflare adds a default Content-Signal: ai-train=yes, search=yes, ai-input=yes.

That last point matters. Our view: if you don't want to signal consent to AI training, set your own content-signal header at the origin before enabling the converter.

Test it:

curl -sI https://www.example.com/page -H "Accept: text/markdown"

What are the risks?

Our view:

No major AI search provider has said, as of 30 Oct 2026, that it ranks or cites pages more because a Markdown version exists. Google explicitly says AI text files aren't needed for AI Overviews and AI Mode (Google).

When does it make sense?

Our view:

• Developer docs: agents and coding assistants read docs heavily, and many docs platforms publish .md copies. This is the strongest case.
• Sites already on Cloudflare: content negotiation is low effort, as long as you check the content-signal default.
• Product and help content used by agents: clean pricing and specs are easier to parse.

It makes less sense for marketing pages, where the HTML is already clean and the main audience is human.

How do you implement .md URLs well?

1. Generate Markdown from your CMS or build step, not by hand.
2. Keep headings, tables and links; drop navigation and boilerplate.
3. Add a rel="alternate" link on the HTML page.
4. List key .md files in llms.txt if you publish one; see llms.txt best practices.
5. Serve with Content-Type: text/markdown; charset=utf-8.
6. Decide on indexing: canonical stays on the HTML page.
7. Log fetches of .md URLs to see whether any agent uses them; see reading server logs for AI bot visits.

How will you know it's working?

Our view: measure usage, not hope. Check server logs for requests to .md URLs or with Accept: text/markdown, and note which user agents send them. If after a few months nothing fetches them, the Markdown copies are cost without benefit, and you can keep them only for docs. If coding assistants or agents do fetch them, compare whether answers about your product get more accurate, using a fixed prompt set.

How does this relate to llms.txt?

They're complementary parts of the same proposal. The llms.txt file is a curated index: a short Markdown file with background and links. The Markdown page versions are what those links point to, so an agent can read the full detail without HTML clutter (llms.txt). Our view: if you publish llms.txt, linking to clean .md copies is more useful than linking to heavy HTML pages.

Exercise

Pick three pages. Generate Markdown versions, add rel="alternate" links, and compare the token count and readability of HTML vs Markdown. Then decide, page by page, whether it's worth maintaining.

Checklist

• Approach chosen (.md URL or negotiation)
• Generated from source, not hand-written
• rel=alternate link added
• Content-signal header set deliberately
• Canonical stays on HTML
• Fetches logged

From MentionX data: which pages agents are likely to want

Method. Read-only aggregate queries over MentionX scans from 22 August to 29 October 2026 (UTC): 2,217 engine runs (2,154 completed) across 18 anonymised projects and six engines.

Our sample doesn't measure Markdown use. It does show which page types engines cite, which suggests where clean text versions might be most useful:

Our view: documentation, product and reference pages are the natural candidates, because agents fetch them for facts and instructions. Marketing pages full of layout and scripts gain least from a Markdown copy compared with simply fixing their HTML.

Limits. A convenience sample of mostly B2B software projects, run through engine APIs, which can behave differently from the consumer apps. Perplexity, Copilot and DeepSeek figures come from a single day (26 October 2026), so treat them as a snapshot, not a trend.

Worked example: Ledgerline adds Markdown to its docs

Ledgerline is a fictional accounts-payable tool. Figures are illustrative.

Ledgerline's developer docs are rendered by a JavaScript framework. Agents fetching them sometimes get incomplete text. The team first makes sure the HTML is server-rendered with full content. Then it adds a .md version of each docs page, generated from the same source files at build time, and links to it from the page head and from its llms.txt file.

The Markdown files contain the same headings, text and code samples as the HTML, without navigation. The team logs requests for .md files by user agent to see who uses them. It makes no claims about citations, but developers report that coding assistants follow the docs more accurately.

Why would an agent want Markdown?

HTML pages carry navigation, scripts, styling and ads alongside the content. An agent fetching a page has to strip all that to find the text, and may get it wrong. Markdown is plain text with simple formatting for headings, lists, links and code. It's compact and easy to parse, so an agent can use more of its context on the content itself.

What should a Markdown version include?

The page title, the main headings in order, all body text, lists, tables and code blocks, and links with their full URLs. Include the date the page was last updated and the canonical HTML URL at the top. Leave out navigation menus, footers, cookie banners and related-content widgets. Keep images as links with alt text.

How do you signal that a Markdown version exists?

Common approaches include a link element in the HTML head pointing to the .md URL, listing .md URLs in llms.txt, and a consistent URL pattern such as adding .md to the page path. None of these is a formal standard that all agents follow. Pick one or two, document them and be consistent.

Should Markdown files be indexed by search engines?

Usually not. You don't want a Markdown copy competing with the HTML page in search results. Add a canonical link header pointing to the HTML page, or block .md files from indexing with an X-Robots-Tag header. Check that this doesn't block agents you want to allow.

What are the costs?

Build-time generation is cheap once set up, but someone must maintain the converter as page templates change. Content negotiation through a CDN feature is easier to switch on but gives you less control over output. Either way, test a sample of pages after each release. Our view: if your HTML is already clean and server-rendered, the gain from Markdown is small, so fix HTML first.

How do you test Markdown output?

Pick ten pages of different types. Compare each Markdown version with the HTML: same headings, same facts, same code, working links. Check tables render correctly, since converters often struggle with complex tables. Re-test after template changes.

Which teams benefit most?

Developer platforms and API companies benefit most, because coding assistants read their docs constantly. Companies with large help centres benefit too, as support agents and assistants answer customer questions from them. A small marketing site with a handful of pages gains little.

How do you measure use?

Log requests for .md URLs, or for pages served as Markdown, by user agent. Watch which agents fetch them and which pages they request. If nobody fetches them after a few months, they may not be worth maintaining. If certain agents fetch them heavily, make sure those pages are complete and current.

How do you handle pages that change often?

If a page changes several times a week, such as a changelog or status page, generate its Markdown on request rather than at build time, or rebuild it on every publish. Add the last-updated date at the top so agents can judge freshness. Stale Markdown is worse than none, because an agent may trust it over the HTML.

What about pages behind a login?

Don't create public Markdown copies of content that's private in HTML. Whatever rules protect the HTML page must apply to the Markdown version too. Check access controls on .md URLs as part of your release tests, since they're easy to forget.

Should you tell users about Markdown versions?

A small "view as Markdown" link on docs pages helps developers who paste docs into their own tools. It also makes the Markdown version easy for your team to check by eye. Some teams also add it to their public changelog so users know it exists. It costs little to add. Try it on a few pages first.

Common questions

Does Markdown help AI citations? There's no public evidence it does. It can help agents read pages accurately.

Is serving Markdown to bots cloaking? Not if the content matches the HTML page. Different content would be a problem.

Do I need both llms.txt and .md files? No. They're separate options; llms.txt can list .md URLs if you have them.

Related

• Paywalls, logins and AI access

Sources

• llms.txt proposal
• Cloudflare, Markdown for Agents docs; announcement
• MDN, Content negotiation
• Google Search Central, AI features and your website

All accessed 30 Oct 2026.