llms.txt: A New Convention, and Whether It's Worth Following

If you have read anything about getting cited by AI this year, you have run into llms.txt. It is a Markdown file at your site root that lists the pages you most want a language model to read.

It takes about twenty minutes to make one. Whether it does anything is a separate question.

What it actually is

The convention is deliberately simple — a file at /llms.txt, plain Markdown, no schema:

# Your Site

> One sentence on what this site is.

## Core pages
- [Home](https://example.com/): what it covers
- [About](https://example.com/about): who is behind it

## Articles
- [Some post](https://example.com/blog/some-post): what it argues

The idea is that a model deciding what to fetch gets a curated map instead of having to guess from your navigation.

What it is not

Three things worth being clear about, because most write-ups skip them:

It is not a standard. No standards body ratified it. No vendor has committed to honoring it. It is a proposal that a number of sites have adopted.

It has no enforcement. Like robots.txt, it is voluntary. Unlike robots.txt, it is not even a long-standing convention that crawlers were built around — so there is no backlog of implementations that already read it.

It is not a substitute for the fundamentals. If your content is client-rendered, an llms.txt pointing at it changes nothing: the crawler still arrives at an empty shell. Fix that first.

Why it is still worth doing

Two reasons, and neither of them is "it boosts your ranking":

1. It costs almost nothing, and it can be generated. Ours is a route handler that reads the same article list the blog index uses. It updates itself when we publish. The marginal cost is zero.

2. It is an unambiguous statement of intent. If a model — or a person — wants to know which pages are your substance rather than boilerplate, you have answered it in one file instead of making them infer it.

That is the whole case. It is cheap, it is legible, and it might help.

How to check yours

Two things to verify, and the first one catches most mistakes:

curl -s https://your-site.com/llms.txt

It should return 200 and actual Markdown. The common failure is a file that exists but contains no links — a rule in our own audit flags exactly this, because a list of headings with no URLs tells a model nothing about where to go.

If you would rather not curl it by hand, the llms.txt tool reports whether the file exists and whether its format holds up, and drafts one from your sitemap.

Then confirm the URLs in it actually resolve. A dead link in llms.txt is worse than no file at all, since you have explicitly pointed at it.

The honest summary

Add it. Spend the twenty minutes. But do not expect it to move traffic on its own — it is a small, cheap signal among the things that actually determine whether you get cited: server-rendered content, a permissive robots.txt, structured data, and having something worth citing in the first place.

That last one is doing most of the work.