AI agents
llms.txt: what it is, how to write one, and whether you need it
llms.txt is a Markdown file at the root of a website that gives AI tools a short summary of the site and links to its most useful pages. It is easy to create and widely recommended, and its value is often oversold. Here is the format, a real example from this site, how to write your own, and an honest view of what it does.
Short answer
llms.txt is a plain Markdown file published at /llms.txt on a website that tells large language models and AI agents what the site is and where its most useful content lives. It holds a title, a summary and annotated links. It is an optional community proposal, not an official standard, and Google Search does not use it.
- Only the H1 with the site's name is required; the summary, details and link sections are optional.
- llms.txt, robots.txt and sitemap.xml do different jobs, and none replaces the others.
- The file is easy to create, and its value is often oversold.
- Generate it from your site's data rather than maintaining it by hand.
- Test it by asking an AI tool to read it.
What is llms.txt?
llms.txt is a plain Markdown file published at /llms.txt on a website that tells large language models and AI agents what the site is and where its most useful content lives. It has a title, a one-paragraph summary and lists of links with short notes. Think of it as a reading list for an AI, where a sitemap is an index for a crawler.
Jeremy Howard proposed llms.txt on 3 September 2024 at llmstxt.org, and published a second version of the proposal in 2026 after two years of adoption. Documentation platforms such as Mintlify and GitBook generate the file automatically, WordPress plugins such as Yoast SEO and AIOSEO can create one, and AI labs publish llms.txt files for their own developer documentation.
What does an llms.txt file look like?
An llms.txt file follows a fixed order: an H1 with the site's name (the only required part), a blockquote with a short summary, optional paragraphs or lists of detail, then sections under H2 headings, each holding a list of links written as [name](url) with an optional colon and note. A section called "Optional" holds links an agent can skip.
| Part | Markdown | Required? | What to put in it |
|---|---|---|---|
| Name | # Site name | Yes | The name of the site or project |
| Summary | > One paragraph | No | What the site is and the key facts needed to understand the rest |
| Details | Paragraphs or lists, no headings | No | How to interpret the links, important notes and rules |
| File lists | ## Section then - [name](url): note | No | Links to the pages that answer real questions, each with a one-line note |
| Optional | ## Optional | No | Secondary links an agent can skip when it is short of context |
An llms.txt example: convertica.org
This is the start of the llms.txt file on this site, which is generated from the same data as the pages. The full file is at convertica.org/llms.txt.
# Convertica
> Conversion for humans and AI agents. Convertica is a conversion rate optimization (CRO) consultancy, founded in 2017, with 2,000+ clients. Verticals served: eCommerce, lead generation, SaaS, affiliate, agencies (white label).
Three offers:
- Free CRO audit: an app that checks a page for people and for AI agents (agent readiness is one of the eight checks). ...
- CRO advisory: ongoing advisory after your free audit. Priced after your audit, places limited. ...
- Full implementation: Convertica's team builds the fixes from your audit for you. Priced after your audit: ...
## Audit, services and pricing
- [CRO audit](https://convertica.org/cro-audit/): The free CRO audit (enter your website and email, and the audit runs live) and a step-by-step guide to running a CRO audit yourself
- [Pricing](https://convertica.org/pricing/): The three offers: the free CRO audit, then CRO advisory or full implementation, both priced after your audit
...Note what the summary does: it states the facts an AI would otherwise have to piece together from several pages, including how pricing works, in plain sentences it can quote.
llms.txt vs robots.txt vs sitemap.xml
llms.txt, robots.txt and sitemap.xml do different jobs. robots.txt sets access rules for crawlers, sitemap.xml lists every page you want indexed, and llms.txt offers AI tools a short, curated summary to read on demand. None replaces the others.
| llms.txt | robots.txt | sitemap.xml | |
|---|---|---|---|
| Job | Summarize the site and point to the best pages | Say which crawlers may access what | List every page for indexing |
| Format | Markdown | Plain-text directives | XML |
| Read by | AI tools and agents that choose to, often on demand | Well-behaved crawlers, before crawling | Search engine crawlers |
| Size | Small enough to fit in an AI's context | Short | Can be very large |
| Status | Community proposal | IETF standard (RFC 9309) | Long-established protocol supported by major search engines |
| Controls access? | No | Yes, for crawlers that honor it | No |
Does llms.txt help SEO or AI search visibility?
llms.txt does not help Google rankings and there is no public evidence that it earns citations in AI search answers. Google states that Google Search ignores llms.txt files, in AI Overviews and AI Mode as well as classic results. Where it does get used is by AI coding and documentation tools, and by some agents that look for it when they visit a site.
Here is what the primary sources say, as of their latest updates:
- Google Search: its guide to generative AI features lists llms.txt under things you do not need to do. It says creating one is fine for other services, and will neither harm nor help your visibility in Search.
- Chrome Lighthouse: the experimental Agentic Browsing audits include an llms.txt check. A missing file is marked not applicable; only a server error when fetching it is flagged.
- Cloudflare's agent readiness scanner: does not check for llms.txt by default; you can add the check when you customize a scan (Cloudflare, April 2026).
- The proposal itself: says the files are used most heavily for software documentation, where coding agents follow them to API references and tutorials.
We have not found a statement from an AI search provider saying it uses llms.txt to decide which sources to cite. If one appears, this page will say so. Until then, treat llms.txt as a small courtesy to AI tools, not a ranking tactic, and spend most of your effort on the pages it points to.
Should your site have an llms.txt file?
Publish an llms.txt file if you have documentation, an API or a product with facts people ask AI tools about, and if you can keep it accurate without manual effort. For most other sites it is a low-priority, low-cost extra: worth doing after the pages themselves are clear.
| Type of site | Priority | Why |
|---|---|---|
| Developer docs and APIs | High | Coding agents use llms.txt files heavily, and Markdown versions of pages save them work |
| SaaS | Medium | Plans, limits and how-to answers are what AI tools get asked; point them to the right pages |
| Online stores | Low | Product data on the page, structured data and feeds matter far more to shopping agents |
| Lead generation and services | Low to medium | A clear summary of what you do, where and how pricing works is cheap to publish |
| Content and affiliate sites | Low | The value is in the articles; make sure crawlers can reach them |
How to create an llms.txt file, step by step
To create an llms.txt file, choose the pages that answer real questions about your business, write a one-paragraph summary of plain facts, group the links under H2 headings with a one-line note each, publish the file at your site root, and generate it automatically so it never goes stale. Then test it by asking an AI tool questions using only the file.
-
Choose the pages that matter
Pick the pages that answer what people ask about you: what you sell, prices or how pricing works, policies, key guides, contact. Leave out tag pages, archives and anything thin. Short beats complete.
-
Write the name and summary
An H1 with your name, then a blockquote with the facts an AI would otherwise have to assemble: what you are, who for, where, and how buying works.
-
Group the links under H2 headings
Use headings people would understand, such as Products, Pricing, Policies, Guides. Put secondary links under an "Optional" heading.
-
Write a useful note for each link
Each note should say what the page answers, in one line. "Returns: 30-day returns, how to start one, refund timing" is useful; "Learn more about our amazing policies" is not.
-
Publish it at the root
Upload it so it loads at
yourdomain.com/llms.txtwith a 200 response. Make sure robots.txt and your firewall do not block it. -
Generate it, do not hand-maintain it
Build the file from your CMS or site data, so a deleted page disappears from it and a price change cannot leave it contradicting your site. Platforms and plugins listed on llmstxt.org can do this.
-
Test it with an AI tool
The proposal's own advice: give an agent only your llms.txt and ask it questions about your business. Fix whatever it gets wrong.
llms-full.txt and Markdown versions of pages
llms-full.txt is a single file holding the full text of a site's documentation, a convention used by documentation platforms and large docs sites. It is not part of the llms.txt proposal, which instead suggests a clean Markdown version of each page at the same URL with .md added.
The second version of the proposal also recommends link relations so tools can find these files: rel="alternate" type="text/markdown" pointing to a page's Markdown version, and rel="describedby" pointing to the llms.txt that covers it, either as HTML link elements or as an HTTP Link header. Cloudflare's developer docs take a related route: they split their llms.txt into one file per product section so each fits in an agent's context, and serve Markdown when an agent asks for it.
For most business sites, neither is needed. They matter when you have a large body of reference content that agents read often.
Common llms.txt mistakes
The most common llms.txt mistakes are listing every page, writing marketing copy instead of facts, letting links go stale, and stating facts that contradict the site. Each makes the file less useful than having none.
- Listing everything. A file that copies your sitemap defeats the purpose. Large sites can split it by section, as Cloudflare does.
- Marketing instead of facts. "The leading provider of innovative solutions" tells an AI nothing it can use.
- Stale links and facts. A hand-written file drifts from the site within months. Generate it.
- Contradicting the page. If llms.txt, the page and the structured data give different prices or policies, none of them can be trusted.
- Instructions aimed at AI. Agents read what you publish as input. Keep the file to facts and links rather than commands to the model.
- Expecting it to fix visibility. If your pages are thin, blocked or hard to read, llms.txt does not change that.
How Convertica generates its llms.txt
The llms.txt on convertica.org is built by the same static site generator as the pages. Its summary comes from the site's own data file, so the founding year, client count, offers and pricing rule match every page. A link is listed only when the page exists in the build, so the file cannot point to a page that is missing. Each link has a one-line note on what the page answers.
It sits alongside the other basics: a robots.txt that names and allows the main AI crawlers, a sitemap, content in the HTML on load, and structured data built from the visible text. The llms.txt is the smallest of those pieces. For the full list, see the AI agent readiness checklist.
Get your site checked for people and AI agents
Convertica's free CRO audit is an app with eight checks, one of them agent readiness: whether an AI agent can read what your page sells, the price and the policies. Enter your website and email, and watch it run live on your page. CRO advisory after it is led personally by Kurt Philip and is priced after your audit. To be cited in AI search answers, see generative engine optimization services.
llms.txt FAQ
What is llms.txt?
llms.txt is a proposed standard for a Markdown file, placed at /llms.txt on a website, that gives large language models and AI agents a short summary of the site and a curated list of links to its most useful pages. Jeremy Howard proposed it in September 2024 at llmstxt.org.
Is llms.txt an official standard?
No. It is a community proposal published at llmstxt.org, open for input through a public GitHub repository. It is not a standard from a body such as the IETF or W3C, and no search engine or AI company is obliged to read it.
Does Google use llms.txt?
Google Search does not. Google's guide to its generative AI features says you do not need AI text files such as llms.txt to appear in them, and that having one will neither help nor harm your visibility in Search. Separately, Chrome's Lighthouse checks for the file in its experimental Agentic Browsing audits.
Where do I put an llms.txt file?
At the root of your site, so it loads at https://yourdomain.com/llms.txt. The current proposal also allows files at a subpath, such as /docs/llms.txt, covering the pages under that path; an agent should use the most specific file that applies.
What is the difference between llms.txt and llms-full.txt?
llms.txt is a short index: a summary plus links. llms-full.txt is a convention used by documentation platforms and sites such as Cloudflare's developer docs, in which the full text of the documentation is put into a single file. llms-full.txt is not part of the llmstxt.org proposal.
Does llms.txt replace robots.txt or a sitemap?
No. robots.txt says which crawlers may access what. A sitemap lists every page you want indexed. llms.txt is a short, curated reading list for AI tools that need context on demand. A site can have all three, and they should agree with each other.
Is llms.txt mandatory?
No. It is optional. Lighthouse treats a missing llms.txt as not applicable rather than as a failure, and Google Search ignores the file. Publish one if it is cheap to keep accurate, but do not expect it to change your rankings or AI citations on its own.