The llms.txt file is a markdown text file, placed at the root of a site (your-site.com/llms.txt), that introduces the site and lists its key pages with a one-line description for each. It is aimed at language models and AI agents. It is a convention proposed on September 3, 2024 by Jeremy Howard, co-founder of Answer.AI, not an official standard.
I'll be blunt about something many articles avoid: Google Search ignores it, and says so in its documentation. OpenAI doesn't mention it in its crawler documentation. Its real readers are AI agents, in particular the coding assistants that browse technical documentation. That is no reason to ignore it, but it is a reason not to expect any SEO effect from it.
Do you need an llms.txt file? Yes, but
It costs ten minutes and carries zero risk: Google states that an llms.txt neither helps nor hurts your visibility in its search. The benefit lies elsewhere, on the side of AI agents and documentation. Worth doing, but after the optimizations whose effect is certain.
- Ten minutes of work, no technical risk
- Read by coding agents and checked by Lighthouse
- Forces you to decide which pages really matter
- Ignored by Google Search, including its AI features
- No documented effect on the sources ChatGPT cites
Priority: after indexing and crawler access Who it's for: documentation, software publishers, content sites
Key takeaways
- llms.txt is a markdown table of contents placed at the root of the domain, at /llms.txt. Only the level 1 heading is required.
- It doesn't replace sitemap.xml: the sitemap lists everything, llms.txt prioritizes. And it has nothing to do with robots.txt.
- Google Search ignores the file: it neither helps nor hurts rankings, including in AI Overviews and AI Mode.
- Its real readers are AI agents: coding assistants and agentic browsing, which Lighthouse has audited since 2026.
- On WordPress, Yoast SEO, Rank Math and AIOSEO generate it, but the descriptions need reviewing.
What is the llms.txt file?
The starting idea is simple. A language model landing on your site has to guess what matters among the menu, the cookie banners, the footers and the three hundred blog posts. The llms.txt file gives it the answer directly: this is who we are, and these are the pages that count.
The format is markdown, deliberately minimal. The specification published on llmstxt.org, whose version 2 dates from August 10, 2026, sets out, in order:
- A level 1 heading with the name of the site or project: the only required element.
- A blockquote summary, introduced by a chevron, that says the essentials in one or two sentences.
- Optional free text to add context.
- Level 2 sections that group annotated markdown links, with a short note after each link.
- By convention, an "Optional" section for secondary links, which an agent can skip if it runs short of space.
Version 2 adds two new features. A page can point to its markdown version with a rel="alternate" type="text/markdown" link, and to the llms.txt that covers it with rel="describedby". And a file placed in a subfolder covers the pages in that subfolder, so documentation hosted under a path can have its own.
llms.txt or llm.txt?
The exact name is llms.txt, with an s: "LLMs", for large language models. "llm.txt" is a common typo. A file with that name matches no convention, and no tool will look for it.
A complete llms.txt example
Here is a correct file for a fictional consulting firm. It includes every element of the specification.
# My Company
> Energy efficiency consultancy for commercial buildings,
> based in Nantes and working across France.
My Company supports owners and managers of offices, shops
and warehouses subject to France's tertiary decree (décret tertiaire).
## Services
- [Energy audit](https://example.com/audit): regulatory assessment and works plan.
- [Consumption monitoring](https://example.com/monitoring): monthly energy tracking and alerts.
- [Tertiary decree](https://example.com/tertiary-decree): OPERAT filing and reduction trajectory.
## Resources
- [Tertiary decree guide](https://example.com/decree-guide): obligations, thresholds and deadlines explained.
- [FAQ](https://example.com/faq): answers to our clients' questions.
## About
- [Our team](https://example.com/team): the consultants and their certifications.
- [Contact](https://example.com/contact): booking a meeting and service area.
## Optional
- [News](https://example.com/news): monthly regulatory updates.Three details make this file good. The summary says who, what and where in two lines. Each link carries a note that says which question the page answers. And the "Optional" section isolates what can be skipped without losing anything.
If you already have a file, check that it is valid. Paste it below: the tool checks the expected structure and flags what is missing.
How to create an llms.txt file
Allow ten minutes for a normal-sized site. The long part isn't technical: it is deciding which pages deserve a place in it.
- List your key pages. Between ten and thirty links, no more. A file that lists three hundred pages no longer prioritizes anything.
- Write the summary. Two sentences that say what you do, for whom, and where. It is the part most likely to be reused word for word.
- Group by section. Services, resources, documentation, about. Explicit headings, not internal category names.
- Describe each link. One line after the link that says what the page contains. This note is what gives the file its value.
- Upload the file to the root, at your-site.com/llms.txt, served as plain text in UTF-8.
To go faster, I put a free llms.txt generator at the top of my homepage. You enter the site name, its URL, a one-sentence description and your key pages (one per line, in the format "Title | /url"); it produces the file in the right format, ready to copy or download. It lays down the skeleton: then add a note after each link, and run the result through the validator above.
To check that the file is served correctly once it is live, one command is enough:
curl -I https://www.your-site.com/llms.txtThe response must start with a 200 code and contain content-type: text/plain. If you get text/html, your host is wrapping the file in a page, and it becomes unusable.
The mistake that makes the file useless
Writing empty descriptions like "our services page". The value of the file lies in the notes: they tell the agent which question each page answers. A useful note looks like "steps and timelines of an energy audit for commercial buildings", not "our offers".
llms.txt on WordPress
The three big SEO plugins can generate the file. Handy, as long as you review what they produce.
- Yoast SEO: the feature is included in the free version. It is enabled in the settings, site features tab, "llms.txt" card. Yoast also offers it in its Shopify version.
- Rank Math: the "LLMS Txt" module is enabled from the plugin dashboard. Its settings let you choose post types, taxonomies and the number of links, set to 100 by default: lower it.
- AIOSEO: generates llms.txt and the llms-full.txt variant, with a choice of post types and categories.
- Without a plugin: upload the file to the root via FTP or your host's file manager. This method gives you the most control over the notes.
Beware of automatic generation: it often produces a list of your latest posts, with the excerpt as the description. The file exists, but it prioritizes nothing. I recommend starting from the generated version, then trimming it down to your twenty useful pages.
On other platforms
Wix also offers to generate the file. On a static site like mine, it is simply a file placed at the root of the published folder. Technical documentation platforms such as Mintlify and GitBook generate it natively: that is where the format was born and where it is most useful.
llms.txt vs robots.txt vs sitemap.xml: the differences
The confusion is constant. These three files live in the same place and do completely different things.
| File | What it says | To whom | Status |
|---|---|---|---|
| robots.txt | What bots are not allowed to crawl | All bots | A 1994 convention, standardized in 2022 (RFC 9309) |
| sitemap.xml | The exhaustive list of pages to index | Search engines | A protocol shared by search engines (sitemaps.org) |
| llms.txt | The pages that matter, and what each one offers | Language models and AI agents | A proposed, unofficial convention |
robots.txt is a prohibition, sitemap.xml an inventory, llms.txt an editorial recommendation. None replaces the others, and publishing an llms.txt without a proper sitemap would be putting the cart before the horse.
Who really reads the llms.txt file in 2026?
The theoretical debate has been going on for two years. Here is what the official sources say, player by player.
| Player | Documented position | Effect for you |
|---|---|---|
| Google Search (AI Overviews and AI Mode included) | Ignores the file, according to its guide to optimizing for generative AI, updated on July 10, 2026 | None, positive or negative |
| Chrome Lighthouse | Experimental "Agentic Browsing" audit that checks the file is present, with no effect on scores | A sign of interest on the browser and agent side |
| OpenAI (ChatGPT) | No mention in its crawler documentation; publishes an llms.txt for its own developer documentation | No documented effect on cited sources |
| Anthropic, Google (Gemini API) | Publish an llms.txt for their developer documentation | Shows the intended use: documentation |
| Coding agents | Read the file to find the right page in a documentation site | The most concrete use case today |
Version 2 of the specification acknowledges this shift: it describes agents that read the llms.txt to find what they need, then follow the links. Its authors state that coding agents use it reliably. For a brochure site or an e-commerce site, however, the benefit remains theoretical: a product catalog is better described with structured data and a product feed than with a text file.
llms.txt and SEO: what impact?
None on Google, and it is written in black and white. Google's guide to optimizing for generative AI states that you don't need to create AI-specific files, markup or markdown versions to appear in Search, AI features included, since Google Search doesn't use them. It adds that maintaining an llms.txt for other services neither hurts nor helps rankings.
On the ChatGPT side, the setting that matters is elsewhere: letting the OAI-SearchBot crawler in through your robots.txt, which I cover in my guide to ChatGPT SEO. So why create an llms.txt? For three reasons that have nothing to do with rankings.
- The agents that read your documentation. When a developer asks an assistant how to use your product, a well-made file points it to the right pages.
- Agentic browsing. Lighthouse now checks that the file is present, and its documentation explains that an agent without llms.txt spends more time exploring the site to understand its structure.
- The exercise itself. Choosing twenty pages out of three hundred and writing down what each one offers is an editorial audit in disguise. I do the exercise with my clients for that reason at least as much as for the file.
What llms.txt will never do
It won't make up for a site missing from search results. Answer engines pull their sources from indexes: if you aren't in them, no text file will put you there. The real levers are in my GEO SEO guide and in the writing rules of AEO. llms.txt sits at the bottom of that list, not the top.
What about llms-full.txt?
The llms-full.txt variant contains the full text of your content, concatenated into a single file, so that an assistant can ingest an entire documentation set in one go. It isn't part of the llmstxt.org specification: it is a practice spread by documentation platforms and some plugins, such as AIOSEO.
My take: useful for technical documentation, useless for a regular site. The file can reach several megabytes, it goes stale with the first update that isn't carried over, and it duplicates content that is already accessible. If you publish product documentation, do it and automate its generation. Otherwise, stick with llms.txt.
The mistakes I see most often
- Listing every page on the site. A file with three hundred links prioritizes nothing: it is a sitemap in disguise, only worse.
- Forgetting the notes. Without the description after each link, the file loses most of its value.
- Serving the file as HTML. It must be returned as plain text. Some hosts wrap it in a template, which makes it unusable.
- Naming it llm.txt. Without the s, no tool will find it.
- Letting it go stale. A table of contents that points to deleted pages is worse than none at all.
- Publishing it while blocking crawlers. The inconsistency is common, and it cancels out the whole effort.
Should you also allow AI crawlers?
This question always comes with the llms.txt one, and it matters far more. Publishing a table of contents for bots you block elsewhere makes no sense.
The main agents to know: OAI-SearchBot (ChatGPT search) and GPTBot (training) at OpenAI, ClaudeBot at Anthropic, PerplexityBot at Perplexity, and Google-Extended, which controls how Gemini uses your content with no effect on Google Search. Each one is managed in your robots.txt.
The decision is yours, and it is legitimate either way. A publisher whose content is the product may want to protect it. A business looking for visibility has every reason to open up. What you shouldn't do is live with this setting without having chosen it, which is the case for most of the sites I audit. If you want to know what AI says about you before changing anything, start with an AI visibility audit.
My recommendation
Do it, but keep it in its place: after checking that your pages can be indexed, that your content answers questions in the first few lines, and that AI crawlers are allowed in. In that order, llms.txt is a good final touch. In the reverse order, it is a decorative file at the root of a site nobody cites. If you want me to prioritize these projects for your site, that is what I do in my work as a GEO consultant.
Frequently asked questions
What is the llms.txt file?
llms.txt is a markdown text file placed at the root of a site that introduces the site and lists its key pages with a short description for each. It is aimed at language models and AI agents. It is a convention proposed in September 2024 by Jeremy Howard, not an official standard.
How do you create an llms.txt file?
List ten to thirty key pages, write a two-sentence summary of what you do, group the links into sections and add a useful note after each link. Then upload the file to your-site.com/llms.txt, served as plain text. The free generator on my homepage builds the skeleton in a few seconds.
What is the impact of llms.txt on SEO?
None on Google. Its guide to optimizing for generative AI states that Google Search doesn't use these files, and that maintaining one neither helps nor hurts rankings, AI Overviews and AI Mode included. Its value lies elsewhere: guiding the agents that read documentation.
Does ChatGPT read the llms.txt file?
OpenAI doesn't mention llms.txt in its crawler documentation, and nothing suggests it influences the sources ChatGPT cites. To appear in its answers, the setting that matters is letting the OAI-SearchBot crawler in through your robots.txt.
How do you add llms.txt to WordPress?
Yoast SEO generates it in its free version, Rank Math through its LLMS Txt module, and AIOSEO also produces the llms-full.txt variant. You can also upload the file to the root via FTP. Always review the generated file: plugins often list your latest posts with no hierarchy.
What is the difference between llms.txt and robots.txt?
robots.txt tells bots what they are not allowed to crawl: it is a prohibition, standardized in 2022. llms.txt does the opposite and recommends the content that matters, in a format language models can read. One protects, the other highlights: they don't replace each other.
Is it llms.txt or llm.txt?
The exact name is llms.txt, with an s, for large language models. llm.txt is a common typo: a file with that name matches no convention and no tool will look for it.
Does llms.txt replace sitemap.xml?
No. The sitemap.xml lists every page to index and is aimed at search engines: it is an inventory. llms.txt selects a handful of pages and explains what each one offers: it is an editorial ranking. A site needs a proper sitemap before it worries about llms.txt.
Can AI read your content?
AI visibility audit: what ChatGPT, Perplexity and Gemini say about your industry, which crawlers get through, and the plan to get into their answers.
Written by
Anthony Courtin
Reader reviews
Did this article help you? Say so, it helps the next readers.
What readers say
Be the first to leave a review.
