llms.txt:What the filereally does

What is llms.txt?

llms.txt is a plain text file located at yourdomain.com/llms.txt, the same place as the familiar robots.txt. It is written in Markdown, a text format people can read without special software and language models handle particularly well. Think of it as a table of contents with an introduction: who runs the website, what it is about, and which pages an AI system should read to understand your business.

The idea addresses a technical problem. A normal web page is full of navigation, ads, scripts and layout. For a language model that is noise, because it can only process a limited amount of text at once. llms.txt is meant to cut through that noise and offer the key content in compact form, optionally with plain text versions of individual pages ending in .md.

One point matters for everything that follows: the file does not control anything. It neither allows nor blocks any crawler, and it does not change what Google indexes. It is an offer to AI systems, which they may or may not take up.

Do ChatGPT, Claude and Google use llms.txt?

Based on everything publicly documented: no, at least not for deciding which websites appear in an AI answer. Google's John Mueller wrote back in June 2025 that no AI system currently uses llms.txt, adding that server logs show the services do not even ask for it. Google has since put this into its own documentation: to appear in AI Overviews and AI Mode, you do not need to create new machine readable files, AI text files or special markup.

The numbers point the same way. Ahrefs analysed server logs from 137,210 domains for May 2026. Around 28 percent of them publish an llms.txt, and about 38,000 files were valid. 97 percent of those files were not requested once during the entire month. Of the few requests, only a small share came from AI search services; the retrieval bots that gather answers for ChatGPT and similar tools accounted for 1.1 percent. The largest single group of requesters were SEO audit tools.

Interestingly, the providers publish files of their own. When we checked on 25 September 2026, the developer documentation of both OpenAI and Anthropic served an llms.txt. That shows the companies find the format useful for their own help pages, for example for coding tools that read documentation. Neither company has said that its crawlers read other sites' llms.txt files for ChatGPT or Claude answers.

llms.txt vs robots.txt: what is the difference?

Both files sit in the root directory and have similar names, but they do completely different jobs. robots.txt tells crawlers which URLs they may fetch. It has been described as RFC 9309 since 2022, and reputable crawlers such as Googlebot follow it. Even robots.txt cannot enforce anything, though: Google itself notes that it is up to the crawler to obey the rules.

llms.txt, by contrast, governs no access at all. It describes what exists and suggests a reading order. If you want to keep AI crawlers away from your site or explicitly allow them, you do that in robots.txt, never in llms.txt. Conversely, even the best llms.txt is useless if robots.txt locks the AI crawlers out or a firewall blocks them. That is why this check is among the first technical steps in our GEO audit.

For more on how ChatGPT reaches content and which crawlers are involved, read our article on ChatGPT SEO.

Where llms.txt reaches its limits

The biggest weakness lies in the concept itself. In llms.txt, a company describes itself, and nobody checks whether the description is accurate. For a system that has to pick the most reliable source among many providers, a self-description carries little weight. It is therefore plausible that AI search services prefer to read the actual pages and what other websites say about you.

The second limit is maintenance. An llms.txt that points to outdated pages or discontinued products is worse than none at all. If you add the file, have it generated automatically or make it part of your regular website upkeep. Google Lighthouse, incidentally, treats a missing file as neutral; only a file whose request triggers a server error gets flagged.

The third limit is expectations. Anyone who adds an llms.txt hoping for more mentions in ChatGPT will, based on everything measured so far, be disappointed. What actually makes the difference in our analyses are other factors: pages that are readable without scripts, a robots.txt that does not accidentally lock out AI crawlers, clearly supported facts on your own website, and mentions on third party sites that AI systems trust. You will find an overview of these levers in our articles on AI SEO and Google AI Overviews.

Whether your work pays off only becomes clear once you measure how often, and in what context, AI systems mention your company. We explain how in measuring AI visibility, and the terms behind it in GEO, AEO and LLMO explained. If your team wants to run these checks itself, our AI training covers how.