# Markdown Versions of Pages for LLMs

> Should you publish Markdown versions of pages for LLMs? The .md URL proposal, Accept: text/markdown negotiation, Cloudflare's converter and the risks.

Source: https://mentionx.ai/learn/technical/markdown-versions-pages-llms  
Author: Dipuck Jones, founder of MentionX  
Updated: 2 November 2026  
Level: Practitioner

**Publishing Markdown versions of pages gives AI agents a clean, low token copy of your content. There are two main approaches: separate `.md` URLs, as proposed by llms.txt, or serving Markdown on the same URL when a client sends `Accept: text/markdown` ([llms.txt](https://llmstxt.org/), [Cloudflare](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/)). Neither is required for Google's AI features ([Google](https://developers.google.com/search/docs/appearance/ai-features)).**

*Facts are from llms.txt, Cloudflare, MDN and Google, accessed 30 Oct 2026. Points marked "Our view" are interpretation.*

> **Key takeaways**
>
> - You can offer Markdown versions of pages either as separate .md URLs or by serving Markdown to agents that ask for it.
> - Markdown versions must match the HTML page's content; anything else risks looking like cloaking.
> - There's no public evidence that Markdown versions raise AI citations; treat them as a convenience for agents.
> - Start with documentation and reference pages, where agents benefit most from clean text.

*Checked on 31 October 2026. "Our view" marks MentionX opinion. Ledgerline is a fictional company used for illustration.*

## What are the two approaches?

| Approach | How it works | Source |
|---|---|---|
| Separate .md URL | `page.html.md` or `page.md`; `index.html.md` for directory URLs | [llms.txt](https://llmstxt.org/) |
| Content negotiation | Same URL; server returns Markdown when `Accept: text/markdown` is requested | [Cloudflare](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/), [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation) |

The llms.txt proposal also recommends link relations so clients can find them: `rel="alternate" type="text/markdown"` pointing to the Markdown version, and `rel="describedby"` pointing to the covering llms.txt file, either as HTML `<link>` elements or an HTTP `Link` header ([llms.txt](https://llmstxt.org/)).

```html
<link rel="alternate" type="text/markdown" href="/learn/guide.md">
```

## How does Cloudflare's Markdown for Agents work?

From Cloudflare's documentation ([Cloudflare](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/)):

- On zones with it enabled, a request with `Accept: text/markdown` makes Cloudflare fetch the HTML from your origin and convert it to Markdown on the fly.
- The response has `Content-Type: text/markdown; charset=utf-8` and `Vary` includes `Accept`, so caches keep HTML and Markdown variants separate.
- Origin headers such as HSTS, CSP, Set-Cookie, CORS and caching headers are preserved. `ETag` and `Last-Modified` are dropped, so conditional requests can't be honored.
- Headers, footers, navigation, scripts and styles are stripped; metadata from `<meta>` tags becomes YAML frontmatter.
- **Content signals:** if your origin doesn't send a `content-signal` header, Cloudflare adds a default `Content-Signal: ai-train=yes, search=yes, ai-input=yes`.

That last point matters. *Our view:* if you don't want to signal consent to AI training, set your own `content-signal` header at the origin before enabling the converter.

Test it:

```bash
curl -sI https://www.example.com/page -H "Accept: text/markdown"
```

## What are the risks?

*Our view:*

| Risk | Why it matters | Mitigation |
|---|---|---|
| Content drift | Hand maintained .md files fall out of sync | Generate from the same source at build time |
| Duplicate URLs in search | .md files may be crawled and indexed | Keep them out of sitemaps; consider `X-Robots-Tag: noindex` on .md |
| Missing context | Pricing tables or widgets may not convert cleanly | Check output for key pages |
| Unintended signals | Default content-signal permits training | Set your own header |
| Effort with no proven payoff | No major engine says it uses them | Start with docs only |

No major AI search provider has said, as of 30 Oct 2026, that it ranks or cites pages more because a Markdown version exists. Google explicitly says AI text files aren't needed for AI Overviews and AI Mode ([Google](https://developers.google.com/search/docs/appearance/ai-features)).

## When does it make sense?

*Our view:*

- **Developer docs:** agents and coding assistants read docs heavily, and many docs platforms publish .md copies. This is the strongest case.
- **Sites already on Cloudflare:** content negotiation is low effort, as long as you check the content-signal default.
- **Product and help content used by agents:** clean pricing and specs are easier to parse.

It makes less sense for marketing pages, where the HTML is already clean and the main audience is human.

## How do you implement .md URLs well?

1. Generate Markdown from your CMS or build step, not by hand.
2. Keep headings, tables and links; drop navigation and boilerplate.
3. Add a `rel="alternate"` link on the HTML page.
4. List key .md files in llms.txt if you publish one; see llms.txt best practices.
5. Serve with `Content-Type: text/markdown; charset=utf-8`.
6. Decide on indexing: canonical stays on the HTML page.
7. Log fetches of .md URLs to see whether any agent uses them; see [reading server logs for AI bot visits](https://mentionx.ai/learn/technical/reading-server-logs-ai-bot-visits).

## How will you know it's working?

*Our view:* measure usage, not hope. Check server logs for requests to `.md` URLs or with `Accept: text/markdown`, and note which user agents send them. If after a few months nothing fetches them, the Markdown copies are cost without benefit, and you can keep them only for docs. If coding assistants or agents do fetch them, compare whether answers about your product get more accurate, using a fixed prompt set.

## How does this relate to llms.txt?

They're complementary parts of the same proposal. The llms.txt file is a curated index: a short Markdown file with background and links. The Markdown page versions are what those links point to, so an agent can read the full detail without HTML clutter ([llms.txt](https://llmstxt.org/)). *Our view:* if you publish llms.txt, linking to clean .md copies is more useful than linking to heavy HTML pages.

## Exercise

Pick three pages. Generate Markdown versions, add `rel="alternate"` links, and compare the token count and readability of HTML vs Markdown. Then decide, page by page, whether it's worth maintaining.

## Checklist

- Approach chosen (.md URL or negotiation)
- Generated from source, not hand written
- rel=alternate link added
- Content-signal header set deliberately
- Canonical stays on HTML
- Fetches logged

## From MentionX data: which pages agents are likely to want

**Method.** Read only aggregate queries over MentionX scans from 22 August to 29 October 2026 (UTC): 2,217 engine runs (2,154 completed) across 18 anonymised projects and six engines.

Our sample doesn't measure Markdown use. It does show which page types engines cite, which suggests where clean text versions might be most useful:

| Cited URL type | Share of 9,750 citations |
| --- | --- |
| Blog or article | 19.6% |
| Homepage | 10.0% |
| Review page | 2.7% |
| Docs | 2.6% |
| Pricing | 1.9% |
| Wikipedia | 1.6% |

*Our view:* documentation, product and reference pages are the natural candidates, because agents fetch them for facts and instructions. Marketing pages full of layout and scripts gain least from a Markdown copy compared with simply fixing their HTML.

**Limits.** A convenience sample of mostly B2B software projects, run through engine APIs, which can behave differently from the consumer apps. Perplexity, Copilot and DeepSeek figures come from a single day (26 October 2026), so treat them as a snapshot, not a trend.

## Worked example: Ledgerline adds Markdown to its docs

*Ledgerline is a fictional accounts payable tool. Figures are illustrative.*

Ledgerline's developer docs are rendered by a JavaScript framework. Agents fetching them sometimes get incomplete text. The team first makes sure the HTML is server rendered with full content. Then it adds a .md version of each docs page, generated from the same source files at build time, and links to it from the page head and from its llms.txt file.

The Markdown files contain the same headings, text and code samples as the HTML, without navigation. The team logs requests for .md files by user agent to see who uses them. It makes no claims about citations, but developers report that coding assistants follow the docs more accurately.

> **Pro move:** Generate Markdown from the same source as the HTML, at build time. Hand maintained copies drift out of date within weeks.

## Why would an agent want Markdown?

HTML pages carry navigation, scripts, styling and ads alongside the content. An agent fetching a page has to strip all that to find the text, and may get it wrong. Markdown is plain text with simple formatting for headings, lists, links and code. It's compact and easy to parse, so an agent can use more of its context on the content itself.

## What should a Markdown version include?

The page title, the main headings in order, all body text, lists, tables and code blocks, and links with their full URLs. Include the date the page was last updated and the canonical HTML URL at the top. Leave out navigation menus, footers, cookie banners and related content widgets. Keep images as links with alt text.

## How do you signal that a Markdown version exists?

Common approaches include a link element in the HTML head pointing to the .md URL, listing .md URLs in llms.txt, and a consistent URL pattern such as adding .md to the page path. None of these is a formal standard that all agents follow. Pick one or two, document them and be consistent.

## Should Markdown files be indexed by search engines?

Usually not. You don't want a Markdown copy competing with the HTML page in search results. Add a canonical link header pointing to the HTML page, or block .md files from indexing with an X-Robots-Tag header. Check that this doesn't block agents you want to allow.

## What are the costs?

Build time generation is cheap once set up, but someone must maintain the converter as page templates change. Content negotiation through a CDN feature is easier to switch on but gives you less control over output. Either way, test a sample of pages after each release. *Our view:* if your HTML is already clean and server rendered, the gain from Markdown is small, so fix HTML first.

## How do you test Markdown output?

Pick ten pages of different types. Compare each Markdown version with the HTML: same headings, same facts, same code, working links. Check tables render correctly, since converters often struggle with complex tables. Retest after template changes.

## Which teams benefit most?

Developer platforms and API companies benefit most, because coding assistants read their docs constantly. Companies with large help centres benefit too, as support agents and assistants answer customer questions from them. A small marketing site with a handful of pages gains little.

## How do you measure use?

Log requests for .md URLs, or for pages served as Markdown, by user agent. Watch which agents fetch them and which pages they request. If nobody fetches them after a few months, they may not be worth maintaining. If certain agents fetch them heavily, make sure those pages are complete and current.

## How do you handle pages that change often?

If a page changes several times a week, such as a changelog or status page, generate its Markdown on request rather than at build time, or rebuild it on every publish. Add the last updated date at the top so agents can judge freshness. Stale Markdown is worse than none, because an agent may trust it over the HTML.

## What about pages behind a login?

Don't create public Markdown copies of content that's private in HTML. Whatever rules protect the HTML page must apply to the Markdown version too. Check access controls on .md URLs as part of your release tests, since they're easy to forget.

## Should you tell users about Markdown versions?

A small "view as Markdown" link on docs pages helps developers who paste docs into their own tools. It also makes the Markdown version easy for your team to check by eye. Some teams also add it to their public changelog so users know it exists. It costs little to add. Try it on a few pages first.

## Common questions

**Does Markdown help AI citations?** There's no public evidence it does. It can help agents read pages accurately.

**Is serving Markdown to bots cloaking?** Not if the content matches the HTML page. Different content would be a problem.

**Do I need both llms.txt and .md files?** No. They're separate options; llms.txt can list .md URLs if you have them.

## Related

- Paywalls, logins and AI access

## Sources

- [llms.txt proposal](https://llmstxt.org/)
- Cloudflare, [Markdown for Agents docs](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/); [announcement](https://blog.cloudflare.com/markdown-for-agents/)
- MDN, [Content negotiation](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation)
- Google Search Central, [AI features and your website](https://developers.google.com/search/docs/appearance/ai-features)
All accessed 30 Oct 2026.

## Frequently asked questions

### Do I need Markdown versions of my pages to appear in Google's AI features?

No. Google says you don't need to create new machine readable files, AI text files or markup to appear in AI Overviews and AI Mode.

### What URL should a Markdown version use?

The llms.txt proposal suggests the same URL with .md appended or the extension replaced by .md, plus a rel=alternate type=text/markdown link.

### What does Cloudflare's Markdown for Agents do?

On enabled zones, it converts HTML to Markdown at the edge when a client sends Accept: text/markdown, and returns Content Type text/markdown.

---

Written and reviewed by Dipuck Jones for MentionX Academy. The HTML version of this page: https://mentionx.ai/learn/technical/markdown-versions-pages-llms
