llms.txt is a plain Markdown file you put at the root of your website (yoursite.com/llms.txt) that gives AI tools a short, curated summary of who you are and links to your most important pages. It is a proposed standard, not an official one, and right now there is no good evidence that having one gets you cited more often by ChatGPT, Google’s AI Overviews or Perplexity.
So should you bother? Our honest answer: yes, if it takes you an hour, because it is cheap, harmless and useful for AI agents and developer tools. No, if someone is selling it to you as the thing that will get your brand into AI answers. That job is still done by the boring work: pages that answer real questions, a clean technical setup, and other sites talking about you. Below is what the file is, what the evidence says, what Google has said on the record, and how to write one in under an hour.
What is llms.txt?.
The idea was proposed by Jeremy Howard of Answer.AI on 3 September 2024, and the spec lives at llmstxt.org. The problem it tries to solve is simple. When a large language model or an AI agent reads a website, it has to wade through navigation, scripts, cookie banners and layout code to find the content. That is slow, costly and error-prone. llms.txt offers a shortcut: one clean, human-readable file that says “here is what this site is, and here are the pages that matter.”
The format is deliberately light. It is Markdown, and the spec asks for these parts, in this order:
- An H1 with the name of the site or project. This is the only required section.
- A blockquote with a short summary of what you do.
- Optional body text with more detail, using any Markdown except headings.
- H2 sections with lists of links, each written as
[name](url)with optional notes after a colon. - An “Optional” section for secondary links an AI tool can skip when it needs less context.
The same proposal also suggests publishing clean Markdown versions of your pages (for example page.md next to page.html), so a tool that follows a link from llms.txt gets text rather than a web page. Most business sites skip that part.
Source: llmstxt.org, spec page checked 26 October 2026.
Is llms.txt the same as robots.txt?.
No, and the confusion causes real mistakes. robots.txt is an instruction file: it tells crawlers what they may and may not fetch, and every major search engine respects it. llms.txt is a summary file: it tells nobody what to do, it just offers a reading list. If you want to block or allow AI crawlers such as GPTBot or ClaudeBot, that still happens in robots.txt (or at your CDN). Putting rules in llms.txt does nothing.
| File | What it does | Who reads it |
|---|---|---|
| robots.txt | Allows or blocks crawling | Search and AI crawlers, by convention |
| sitemap.xml | Lists every URL you want found | Search engines |
| llms.txt | Curated summary and key links | Some AI agents and developer tools |
| Schema (JSON-LD) | Describes entities on a page | Search engines, for rich results |
Does llms.txt help you get cited by AI?.
On the evidence available today, no measurable amount. Three things are worth knowing.
1. The largest study we found shows no link
SE Ranking looked at almost 300,000 domains. About 10.13% had an llms.txt file. When they built a model to predict how often a domain gets cited by AI, removing the llms.txt variable actually made the model more accurate, which suggests the file was noise rather than a signal. It is observational data, not a controlled test, and they note many CMS plugins now generate the file automatically. But it is the best large-scale read we have, and it points one way.
Source: SE Ranking, “llms.txt: What the Data Says”, September 2026.
2. Google has said, repeatedly, that Search ignores it
In June 2025, Google’s John Mueller wrote that “no AI system currently uses llms.txt” (reported by Search Engine Roundtable). Google’s own documentation is now explicit. Its guide to AI features in Search says you don’t need to create new machine-readable files, AI text files or Markdown to appear in Google Search, and that doing so “will neither harm nor help” your visibility, because Google Search ignores them.
Source: Google Search Central, AI optimization guide, last updated 10 July 2026.
3. But browsers and agents are starting to look for it
Here is the twist. Chrome’s Lighthouse added an experimental “Agentic Browsing” audit that checks whether your llms.txt can be fetched. A missing file is marked “not applicable” because the file is optional, but a file that returns a server error gets flagged. That audit is about AI agents browsing on a user’s behalf, not about Google rankings. It is a sign of where the file may matter next: tools that act, rather than search engines that rank.
Source: Chrome for Developers, Lighthouse llms.txt audit, last updated 5 May 2026.
llms.txt won’t move your rankings or your AI citations today. It is a cheap, harmless file that helps AI agents and developer tools read your site, and that use case is growing. Write one in an hour, then spend the rest of your week on the work that actually earns citations.
What actually gets you cited by AI search?.
If the goal behind the question is “how do we show up when someone asks ChatGPT or Google’s AI Mode about our category,” the file is a footnote. In our work on GCC sites, the things that move AI visibility are the same things that move organic search, done with more discipline:
- Answer the real sub-questions. AI tools break a buyer’s prompt into smaller searches and cite the pages that answer those smaller questions best. A page titled with the question, answered in the first two sentences, beats a clever headline.
- Be crawlable without JavaScript. Not every AI crawler runs scripts. If your content only appears after the page loads in a browser, it may be invisible. Server-side rendering fixes that.
- Make your entity unambiguous. Consistent company name, address and description across your site, Google Business Profile, LinkedIn and directories, backed by Organization schema.
- Get mentioned elsewhere. Reviews, press, partner pages and genuine lists. AI answers lean on what other sites say about you, not only what you say about yourself.
- Write in the language your buyers search in. In the Gulf that often means Arabic and English, each written natively rather than machine-translated.
We go deeper on each of these in our guide to getting cited by ChatGPT and AI search, and in the short answer on how AI assistants choose sources to cite.
How to write an llms.txt file in under an hour.
If you decide it is worth the hour, do it properly. A sloppy file is worse than none, because it can hand an agent the wrong summary of your business.
Write the summary
An H1 with your company name, then a two-line blockquote: what you do, for whom, where. Plain words, no slogans.
Pick 10–25 links
Services, pricing approach, key guides, case studies, contact. Group them under H2s. Each link gets one line explaining what is on it.
Publish and test
Serve it at /llms.txt as plain text, check it returns 200 (not a redirect or error), and add a reminder to update it when pages change.
A minimal example for a services business:
# Example Company > Example Company is a B2B logistics software provider in Dubai serving > freight forwarders across the GCC. English and Arabic support. ## Product - [Platform overview](https://example.com/platform): what the software does - [Integrations](https://example.com/integrations): supported ERPs and carriers ## Company - [About](https://example.com/about): team, offices, licences - [Contact](https://example.com/contact): sales and support ## Optional - [Blog](https://example.com/blog): industry articles
For a live reference, you can read ours at kandomarketing.com/llms.txt, or a documentation-heavy example like Anthropic’s docs llms.txt.
Common mistakes we see
- Copying the whole sitemap into it. The point is curation. Hundreds of links defeat the purpose.
- Marketing language. “Award-winning, world-class solutions” tells an AI nothing. Say what you sell and to whom.
- Letting it go stale. If your services or offices change and the file doesn’t, you are feeding agents an out-of-date description of your business.
- Blocking it by accident. A firewall rule, a redirect to the homepage or a 500 error makes the file useless and, in Lighthouse’s new audit, gets flagged.
- Putting crawl rules in it. Access control belongs in robots.txt.
Should GCC businesses add Arabic to llms.txt?.
The spec doesn’t address languages, so there is no standard answer. Our practical view for a bilingual Gulf site: keep one file, write the summary in English, and add an H2 that links to your Arabic section and its key pages, with the link names in Arabic. That way an agent working for an Arabic-speaking user can find the right pages without you maintaining two files that drift apart. If your Arabic site lives on its own path (for example /ar/), the spec allows a separate file at /ar/llms.txt, which tools should prefer for URLs under that path.
The bigger point for the Gulf is the one in the previous section: Arabic visibility in AI answers comes from good Arabic pages, not from a file that lists them. Our answer on Arabic content and AI search in the Gulf covers that in more detail.
Want to know if AI search can see you?
We audit GCC sites for both front doors of search: Google rankings and AI answers. You get the short list of fixes that matter, in order.
Frequently asked questions.
Is llms.txt an official standard?
No. It is a community proposal published at llmstxt.org in September 2024. No standards body has adopted it, and we are not aware of any major AI company publicly committing to use it for answers or rankings.
Does Google use llms.txt?
Google Search does not. Its AI optimization guide says Search ignores AI text files and that adding one will neither help nor harm your visibility. Separately, Chrome’s Lighthouse has an experimental agentic-browsing audit that checks whether the file can be fetched.
Can llms.txt hurt my SEO?
Not according to Google’s documentation. The only real risk is a badly written or outdated file giving AI agents the wrong picture of your business.
Where do I put llms.txt?
At the root of your domain, so it loads at yoursite.com/llms.txt as plain text. The spec also allows files in sub-paths, such as /docs/llms.txt, that cover only that section.
What is llms-full.txt?
Some sites publish a second file with the full text of their key pages in one place. You will mostly see it on software documentation sites. It is not part of the core llms.txt proposal, which instead suggests Markdown versions of individual pages.
Should I block AI crawlers instead?
That is a separate decision made in robots.txt. If you want to appear in AI answers, blocking the crawlers that feed them works against you. If you have content you don’t want used, block it there, not in llms.txt.
So, what is llms.txt? A tidy, optional summary of your site for AI tools. Add one, keep it accurate, and don’t expect it to do the work of good pages. If you want help with the part that does move the needle, our SEO and AI search service covers rankings and AI citations in English and Arabic, and the Kando glossary defines the terms used above.