Visibility in AI answers
Do you need llms.txt? What it is and what the evidence says
You do not need llms.txt for Google: Google Search ignores it, so it neither helps nor harms. OpenAI, Anthropic and Perplexity publish llms.txt for documentation, but we found no statement that their crawlers read other sites' files for answers. On 137,210 sites using its analytics, SEO vendor Ahrefs found that 97% of llms.txt files got no requests in May 2026.
What is llms.txt and where does it come from?
llms.txt is a plain text file in Markdown at the root of a website, such as your-site.com/llms.txt. It gives AI agents a short, curated overview of the site: its name, a summary and links to the main pages. It is a proposed format, not an official standard.
Jeremy Howard published the proposal at llmstxt.org on 3 September 2024 and released version 2 on 10 August 2026. It is a proposal by one author, and the llmstxt.org page itself calls it a proposal. llms.txt also does not block anything, so it is not a new robots.txt.
| File | What it does | Who it is mainly for | Does Google Search use it? |
|---|---|---|---|
robots.txt | Tells automated tools what access to the site is acceptable | Search engine and AI crawlers | Yes |
sitemap.xml | Lists all the pages you want indexed | Search engines | Yes |
llms.txt | Gives a short, curated overview with links | AI agents, today mostly coding agents reading developer documentation | No: Google says its search ignores the file |
What do Google and Chrome say about llms.txt?
Google says its search does not need llms.txt. Its guide to generative AI features in Google Search was last updated on 10 July 2026. It says Google Search ignores these files, so creating one will "neither harm nor help" your visibility or rankings. The guide lists AI text files such as llms.txt among tactics you can ignore.
Google added this note on 15 June 2026 and said it is fine to keep such files for other services. Chrome's Lighthouse tool is a different product. It checks llms.txt for AI agents in the browser (page updated 5 May 2026). It marks a missing file "Not Applicable" and flags only a server error.
Neither page promises any effect on whether ChatGPT or Google cites you. For what else Google calls unnecessary, see AEO vs GEO vs SEO: what's the difference.
| Company or product | Publishes its own llms.txt | Says it reads llms.txt on your site |
|---|---|---|
| Google Search | Not applicable | No. Its guide says Google Search ignores the file |
| Chrome Lighthouse | Not applicable | Checks the file as optional; no file means "Not Applicable", not an error |
| Google, Gemini API | Yes, for developer documentation: ai.google.dev/gemini-api/docs/llms.txt | No statement found |
| OpenAI | Yes, for developer documentation: developers.openai.com/llms.txt. openai.com/llms.txt returns 404 | No statement found. Its crawler overview mentions llms.txt only as the index of its own documentation |
| Anthropic | Yes, for API and Claude Code documentation: platform.claude.com/llms.txt. anthropic.com/llms.txt returns 404 | No statement found on its page about how it crawls the web |
| Perplexity | Yes, for API documentation: docs.perplexity.ai/llms.txt | No statement found on its crawler page |
Do ChatGPT, Claude and Perplexity read llms.txt?
We found no statement from any assistant that its crawler reads other sites' llms.txt files for answers. That does not prove they never do: the companies simply do not confirm it. The only usage data we found is a study by Ahrefs of 137,210 domains in its own Web Analytics product that had traffic in May 2026. It is not a sample of the whole web:
- 28% of these domains publish a file. Ahrefs calls this an upper bound: its customers are more technical than the web as a whole.
- 97% of the files got no requests at all in May 2026.
- Named AI bots made 19.5% of the requests, led by GPTBot, which collects training data, and Claude-Code, Anthropic's coding tool.
- AI bots made zero requests for files that do not exist. They do not go looking for one.
A request is not the same as using the file in an answer, so these figures are a ceiling. The bot shares rest on the 3% of files that got any requests, about 1,100 domains. And Ahrefs sells SEO tools: this is a vendor study, not an independent one.
Who is llms.txt actually useful for?
llms.txt is most useful for software documentation, where coding agents look for the right page. The AI companies use it that way themselves: OpenAI, Anthropic, Perplexity and Google publish llms.txt for their developer documentation, while openai.com/llms.txt and anthropic.com/llms.txt return 404.
The proposal says the format also suits a business that describes its structure and policies. It does not claim any measured effect on AI answers. For a small company in the Baltics, we have no evidence that even a careful llms.txt changes what ChatGPT says about you. Is it worth it? Only if you will keep it current.
What does an llms.txt file look like and how do I make one?
An llms.txt file is a short Markdown text. It has a heading with your company name, a one-paragraph summary, and sections with links to your main pages, one line each. It should say nothing that your pages do not say. You can make one in three ways:
- By hand: a UTF-8 text file uploaded to the root of the site. It is the only way to know every word in the file.
- With a WordPress plugin: Yoast SEO and AIOSEO both work as llms.txt generators. AIOSEO switches the file on by default, so read what it produced.
- With a website builder: Wix creates the file automatically, according to Ahrefs and llmstxt.org. We have not checked other builders.
Keep it short: a small index with links, not the text of every page. Our own learnxmi.com/llms.txt is longer (22,188 bytes, 108 lines, 51 links) because it also lists our prices and page summaries. A small company site does not need that much.
The template below is for a fictional company. Its name keeps the Latvian letters on purpose: they are the first to break when the server sends the wrong encoding. The file lives at https://your-site.com/llms.txt and should be sent as text/plain; charset=utf-8.
| Line in the file | What it is for |
|---|---|
# SIA Piemērs | Heading with the company name, as on the website and in the register. The only required part |
> SIA Piemērs designs and builds timber houses in Riga. We work in Latvia and give the project price before work starts. The website is in Latvian, English and Russian. | Summary written as a blockquote: who you are, what you do, where and for whom. Facts, no adjectives |
Reg. No. 40000000000, Riga. Contact: info@piemers.example, +371 20000000. | Optional notes without a heading: registration, contacts, languages |
## Services | A section heading |
- [Timber house design](https://piemers.example/en/services/design): from sketch to building permit, timelines and stages | A link with one line about what the page contains |
- [Construction](https://piemers.example/en/services/construction): materials, warranty, order of work | One link per page, only pages of your own site |
## Prices | Prices get their own section |
- [Prices and packages](https://piemers.example/en/prices): published prices with the date they were checked | A link to the page where prices live, so the file does not repeat numbers that can go out of date |
## Answers | Pages that answer buyers' questions |
- [How much does a timber house cost](https://piemers.example/en/answers/timber-house-cost): price ranges by floor area | Answer pages are the ones an assistant can quote |
## About | Company and contacts |
- [About us](https://piemers.example/en/about): team, company details | Details that confirm who you are |
## Optional | By convention, secondary pages go here |
- [Privacy policy](https://piemers.example/en/privacy) | A link may have no description |
What mistakes do people make with llms.txt?
Most mistakes are in the content, not the format. These five come up most often:
- Demo text from the theme. According to AIOSEO's documentation (updated 14 August 2026), its file takes the Site Title and the site's Tagline by default. If both were left over from a demo theme, the file describes the theme, not your business.
- No
charset=utf-8. Some hosts send.txtfiles astext/plainwithout a charset. Browsers usually guess the encoding right, but other clients that do not guess the way a browser does can show ā, š, ž or € as garbage. It happened to us on Hostinger. We fixed it on 13 October 2026 by serving the file through a route that sendstext/plain; charset=utf-8. - Old prices. Link each price to the page where it lives, or add the date you checked it. Update the file together with the price page.
- Links to redirects and 404 pages. Every link should return status 200 and point to the canonical address of the page.
- Advertising instead of facts. "The best in Riga" gives an agent nothing to use. Write what you do, where, for whom and at what price.
To check the encoding, run curl -sI https://your-site.com/llms.txt and look at the Content-Type line.
Can llms.txt hurt my site?
Not in Google: Google says the file will neither harm nor help your visibility in Google Search. Harm can only come from what is in the file: old prices, demo text, broken links or the wrong encoding. The real cost of llms.txt is not writing it but keeping it current.
Ahrefs adds that agents trust the file, so an outdated or tampered one can mislead them. It advises treating the file like code: keep versions, limit who can edit it, and link only to your own pages.
What should I do before llms.txt?
Before llms.txt, make sure AI crawlers can reach and read your pages and that the pages are indexed. OpenAI and Google document crawler access and indexing as conditions for appearing in their answers; neither says that about llms.txt. For the checks, see why ChatGPT doesn't recommend your business.
Next, write pages that answer your buyers' questions, with numbers in the text. Take our client's online store hybridenergy.lt. From 28 July to 10 October 2026, 7 of its 10 pages with the most impressions in Google's AI features were buyer guides. For more on this approach, see AEO vs GEO vs SEO: what's the difference.
Then measure: ask your buyers' questions in ChatGPT and Gemini, each twice in a new chat, and repeat after 1–2 weeks. This is our work in visibility in AI answers, from €200 per month. For what drives the price, see how much GEO costs.
How we treat llms.txt
We treat llms.txt as optional. For clients we first check crawler access, the page text and indexing. Then we write pages that answer buyers' questions and measure before and after with the same questions. We do not guarantee a place in AI answers. We guarantee a before-and-after measurement.
If you want an llms.txt anyway, the template and the checks on this page are enough to make one yourself. Our own file is at learnxmi.com/llms.txt.
The method and dated results: visibility in AI answers. If you would rather start with a conversation, in a free 90-minute consultation we look together at what ChatGPT answers to your buyers' questions today.
Sources
- llmstxt.org. The /llms.txt file, version 2 (Jeremy Howard; published 3 September 2024, changed 10 August 2026) (14 October 2026)
- llmstxt.org. Changes in version 2 (August 2026) (14 October 2026)
- Google Search Central. Optimizing your website for generative AI features on Google Search, section on what you don't need to do (updated 10 July 2026) (14 October 2026)
- Google Search Central. Search updates: new guide on generative AI features (15 May 2026) and clarifying guidance on llms.txt files (15 June 2026) (14 October 2026)
- Google Search Central. AI features and your website (updated 10 December 2025) (14 October 2026)
- Chrome for Developers. Lighthouse agentic browsing audit: llms.txt (updated 5 May 2026) (14 October 2026)
- OpenAI. llms.txt for the OpenAI developer documentation (openai.com/llms.txt returns 404) (14 October 2026)
- OpenAI. Overview of OpenAI Crawlers (llms.txt mentioned only as the documentation index) (14 October 2026)
- Anthropic. llms.txt for the Claude API documentation (anthropic.com/llms.txt returns 404) (14 October 2026)
- Anthropic. llms.txt for the Claude Code documentation (14 October 2026)
- Anthropic Help Center. Does Anthropic crawl data from the web, and how can site owners block the crawler? (14 October 2026)
- Perplexity. llms.txt for the Perplexity API documentation (14 October 2026)
- Perplexity. Perplexity Crawlers (14 October 2026)
- Google. llms.txt for the Gemini API documentation (a script is first sent through a Google sign-in check; opens in a browser without an account) (14 October 2026)
- Ahrefs. We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read (Louise Linehan, 15 June 2026; vendor study) (14 October 2026)
- AIOSEO. How to Create an LLMS.txt Using All in One SEO (updated 14 August 2026) (14 October 2026)
- Yoast. How to enable llms.txt with Yoast SEO (14 October 2026)
- learnxMI: our own llms.txt (22,188 bytes, 108 lines, 51 links; served as text/plain; charset=utf-8) (14 October 2026)
- hybridenergy.lt (our client): Google Search Console, "Generative AI features" report, 28 July to 10 October 2026 (12 October 2026)
Frequently asked questions
What is llms.txt?
llms.txt is a Markdown text file at the root of a website. It gives AI agents a short overview: the site's name, a summary and links to the main pages with one line each. Jeremy Howard proposed it in September 2024 and released version 2 in August 2026. It is a proposed format, not an official standard.
Is llms.txt worth adding to a small company website?
It is optional, and worth adding only if you will keep it current. Google Search does not use the file, and we found no statement from an assistant that it reads other sites' files for answers. A short, accurate file costs little. An out-of-date one gives wrong facts to any agent that reads it. Pages that answer buyers' questions come first.
Does ChatGPT read llms.txt?
We found no statement from OpenAI that its crawlers read llms.txt on other sites. OpenAI's crawler overview mentions llms.txt only as the index of its own developer documentation, and OpenAI publishes its own file only for that documentation. Ahrefs saw GPTBot, OpenAI's training crawler, request llms.txt files, but a request does not show that a file is used in answers.
Does Google use llms.txt in AI Overviews?
No. Google's guide to generative AI features says Google Search ignores llms.txt files, including in its generative AI features. So they neither help nor harm your visibility or rankings. You need no special AI text files or markup to appear in AI Overviews. Google clarified this on 15 June 2026 and says it is fine to keep the file for other services.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what access to your site is acceptable. sitemap.xml lists the pages you want indexed. llms.txt is a short, curated overview with links for AI agents. It does not block or allow anything, so it cannot replace robots.txt. The two files do different jobs and can sit side by side.
How do I create llms.txt?
Write a UTF-8 text file with a heading for your company name, a one-paragraph summary and sections of links to your main pages. Upload it to the root of your site. WordPress plugins such as Yoast SEO and AIOSEO can generate one. Whichever way you choose, open the result and read it before you rely on it.
How long does it take and what does it cost?
We have no measured time to give you. It depends on how many pages you have and whether their texts are ready. A plugin creates a file in a few clicks, but you still need to read what it wrote. The real cost is upkeep: the file has to change whenever your prices or pages change.
Can an outdated llms.txt do damage?
Not in Google Search: Google says the file neither harms nor helps. Harm can only come from the content: prices that are out of date, demo text from a theme, broken links or the wrong encoding. Any agent that does read such a file gets wrong facts about you, so no file is better than a bad one.
What is llms-full.txt?
llms-full.txt is a larger file with the full text of a site's pages, which some generators create next to llms.txt. AIOSEO, for example, offers it as a separate option. The llmstxt.org proposal does not define it, and a small company site does not need it: the point of llms.txt is a short index.
How do I check my llms.txt?
Run curl -sI https://your-site.com/llms.txt and look for status 200 and text/plain; charset=utf-8. Read the file for demo text and old prices, and open every link. Give an AI assistant only the file and ask it about your company. Your server logs show who requests the file; they cannot show whether anyone cites it.
See what ChatGPT says about your industry today
In a free 90-minute consultation we look together at what ChatGPT answers to your buyers' questions and name the first fixes.