What Is llms.txt? Should Your Business Website Have One in 2026?

Someone told you ChatGPT reads a file called llms.txt, your SEO plugin offered to generate one, or a Lighthouse report listed it. The question is the same: does a business website need one, and will it get the site cited by AI answer engines?

This guide is for business owners, marketers and freelancers who run sites. It explains the format, with pepochat.com's own file as the example, what each AI and search company says about reading it, how it differs from robots.txt and a sitemap, how to measure it and when to skip it. For the wider job of getting cited, start with our checklist for getting cited by ChatGPT and Perplexity.

pepochat.com is our own site, for PepoChat, the AI support agent we build, and has published an llms.txt since 17 September 2026. Every company statement below was read in that company's own documentation on 22 October 2026.

What is llms.txt?

llms.txt is a plain-text file published at /llms.txt on a website that gives AI agents a short summary of the site and a curated list of links to its most useful pages, written in Markdown. Markdown is a plain-text format in which simple symbols mark structure, such as a hash sign for a heading and a dash for a list item, so people and programs can read the same file.

Jeremy Howard published the proposal on llmstxt.org and on Answer.AI's blog on 3 September 2024. Version 2, dated August 2026 and last modified on 10 August, adds a way for agents to find each page's Markdown copy and defines what a file in a subfolder covers. The llms.txt proposal calls itself just that, a proposal, open to community input on GitHub, while robots.txt became an IETF standard in 2022.

The reasoning is practical. Web pages wrap their content in navigation, ads and scripts, and most sites are too large for a language model's context window, so a short file that says what matters, and where the detail lives, saves an agent time and tokens.

According to the proposal, the files are used most heavily for software documentation, where coding agents follow them to API references, and they are mostly read on demand while an agent helps someone, rather than for model training. Its author reports thousands of sites publishing one. Wix generates one for eligible sites, Yoast SEO does so on WordPress, and documentation platforms such as Mintlify create them automatically. On 22 October 2026, the developer docs of OpenAI, Anthropic, Google's Gemini API and Perplexity all served one.

What goes in an llms.txt file?

An llms.txt file holds, in this order, an H1 heading with the site's name, a blockquote summary, optional notes, and H2 sections that list links with a short note on each; the H1 heading is the only required part. Here is a file for a made-up bakery:

# Northwind Bakery

> Northwind Bakery is a family bakery in Portland, Oregon, selling sourdough
> bread and celebration cakes. Cake orders need 7 days' notice; delivery within
> 10 miles. Prices as of October 2026.

Open Tuesday to Sunday, 7 am to 3 pm. We do not ship outside Oregon.

## Ordering

- [Custom cakes](https://example.com/cakes.md): sizes, prices and lead times
- [Delivery and pickup](https://example.com/delivery.md): zones, fees, cut-off times

## Policies

- [Cancellations and refunds](https://example.com/refunds.md): the 48-hour rule

## Optional

- [Our story](https://example.com/about.md)

Each list item is a Markdown link, optionally followed by a colon and a note. A section headed Optional holds links an agent can skip when it needs a shorter read; since version 2 that is a convention, not a rule.

Format as set out in version 2 of the llms.txt proposal, read on 22 October 2026.

The file belongs at the site root, so it loads at https://example.com/llms.txt. Version 2 also allows one in a subfolder, such as /docs/llms.txt, covering only the pages under that path, and suggests clean Markdown copies of important pages at the same address plus .md. A page can point to its copy with a rel="alternate" link and to its llms.txt with rel="describedby". Documentation platforms generate the copies; most business sites skip them.

The proposal's own advice is short: plain language, a brief note on every link, no unexplained jargon, and a test in which an agent gets only your llms.txt and answers questions from it. Write the summary like an FAQ answer, fact first with a number and a date, as in our guide to writing an FAQ page.

What does a real llms.txt look like? pepochat.com's file

The llms.txt on pepochat.com, live since 17 September 2026, opens with the product name and a one-paragraph summary, then lists 7 key facts, 7 site pages, more than 100 guides and 30 comparison pages under four H2 headings. Here is a shortened excerpt:

# PepoChat

> PepoChat is an AI customer support agent for websites. It answers visitors
> from a private knowledge base built from your own documents and web pages, (...)

Product: https://pepochat.com
App: https://app.pepochat.com
Company: PepoCloud LLC
Support: [email protected]

## Key facts

- Install: one script tag (https://widget.pepochat.com/widget.js with a data-organization-id attribute); snippets for HTML, JavaScript, React and Next.js.
(6 more facts)

## Pages

- [Pricing](https://pepochat.com/pricing): Free, Starter and Growth plans, FAQ
(6 more pages)

## Guides

- [How to Write an FAQ Page That Google and ChatGPT Can Use](https://pepochat.com/blog/how-to-write-faq-page)
(more than 100 more guides)

## Alternatives (side-by-side comparisons with PepoChat, prices verified October 2026)

- [The Best Tidio Alternative in 2026 (Tidio vs PepoChat)](https://pepochat.com/alternatives/tidio-alternative)
(29 more comparisons)

In October 2026 the file had about 170 lines and 20 KB, served as UTF-8 plain text, and it grows with every post; the first version, in September, had 30 lines. Its summary repeats prices and reply allowances, so the file has to change whenever pricing does.

We maintain it by hand. Our publishing checklist adds each new post to the site's post registry, which feeds the XML sitemap automatically at build time, and adds one line for the post to llms.txt, a static file someone has to edit. The file has changed in about 20 commits, nearly all of them new lines for new posts, and a missed edit raises no error: the page simply never appears in the file.

Measured against version 2 of the proposal, our file bends three suggestions. The key facts sit under an H2 heading, where the proposal expects only link lists; the guide entries have titles but no notes; and every link points to an HTML page, because the site publishes no Markdown copies. Agents can still read it, but nothing warned us: the only automated check we know of, in Lighthouse, tests just that the file loads.

Does ChatGPT, Perplexity, Claude or Google read llms.txt?

As of 22 October 2026, no major answer engine has said it reads llms.txt to rank or cite pages: Google says Google Search ignores the file, and OpenAI, Perplexity, Anthropic and Microsoft do not mention it in their crawler documentation.

Google's guide to optimizing for generative AI features, last updated on 10 July 2026, lists llms.txt among the things you can ignore: Google Search, including its generative AI features, does not use such files, and keeping one will "neither harm nor help your site's visibility or rankings in Google Search". To appear in AI Overviews, the guide says, a page must be indexed and eligible for a snippet.

Another Google product does look for the file. Chrome's Lighthouse llms.txt audit, one of its agentic browsing audits, last updated on 5 May 2026, flags a site only when the server errors on the file, and marks a missing file as not applicable because the file is optional. It checks readiness for AI agents that browse your site; it is not a Search signal.

OpenAI's crawler page describes OAI-SearchBot for ChatGPT search, GPTBot for training and ChatGPT-User for pages fetched during a conversation. Perplexity's covers PerplexityBot and Perplexity-User; Anthropic's, dated 7 April 2026, covers ClaudeBot, Claude-User and Claude-SearchBot; and Bing's lists Bingbot, whose index Copilot shares, according to Bing's webmaster guidelines. All of them explain robots.txt controls, and none mentions llms.txt.

Read in each company's crawler or webmaster documentation and Google's AI optimization guide on 22 October 2026; llms.txt files checked the same day.

Publishing is not reading: those files serve coding agents using each company's API, not its search product.

Answer engines search their own index, built by the search crawlers in the table, and cite the passages they retrieve, as our RAG explainer for non-engineers describes. An assistant fetching pages for a user could open your llms.txt if a person pastes its address, but no company documents that it looks for the file on its own. Your logs can tell you.

How is llms.txt different from robots.txt and a sitemap?

A robots.txt file tells crawlers which paths they may fetch, an XML sitemap lists the pages you want search engines to find, and llms.txt is a curated reading list for AI agents; only the first two are formats that search engines say they use.

robots.txt is a plain-text file at the root of a site that tells each crawler which paths it may and may not fetch. Martijn Koster defined the protocol in 1994, and the IETF published it as RFC 9309, a Standards Track document, in September 2022. The RFC puts the file at /robots.txt in UTF-8, tells crawlers not to trust a cached copy for more than 24 hours, and warns that its rules are requests, not security.

An XML sitemap is a file that lists a site's URLs, optionally with the date each last changed, so crawlers can find every page. The sitemaps.org protocol, version 0.90, is supported by Google, Yahoo and Microsoft; one sitemap holds up to 50,000 URLs and 50 MB uncompressed, and listing a page does not guarantee that it gets indexed.

Standards and support as stated on rfc-editor.org, sitemaps.org and llmstxt.org, read on 22 October 2026.

llms.txt has no rules, so it can neither block a crawler nor invite back one you disallowed. It cannot opt you out of AI training either; that takes robots.txt tokens such as GPTBot, ClaudeBot and Google-Extended, listed in full in the checklist linked above. Our own robots.txt allows all crawlers everywhere except /api/ and names seven AI and search crawlers explicitly, from GPTBot to Bingbot, so that a future blanket rule cannot shut them out by accident.

Nor does llms.txt replace a sitemap: the proposal itself says a sitemap lists every indexable page, while llms.txt is a curated overview. And it cannot rescue content an agent cannot read, such as answers that appear only after scripts run, as our guide to why a chatbot says "I don't know" shows.

How can you tell whether AI crawlers read your llms.txt?

Only your server or CDN logs can show whether an AI crawler requested your llms.txt, because Google Analytics 4 automatically excludes known bots, and its tracking tag never runs on a plain-text file anyway.

A user agent is the name a browser or crawler sends with every request, such as GPTBot or bingbot, and it is how your logs tell visitors apart. Five steps answer the question for your own site:

1. Get 30 days of raw access logs from your host or CDN.
2. Keep the requests for /llms.txt, plus /robots.txt and your home page as a comparison.
3. Count them by user agent. Look for OAI-SearchBot, GPTBot, ChatGPT-User, PerplexityBot, Perplexity-User, ClaudeBot, Claude-User, Claude-SearchBot and bingbot. Google-Extended never appears, because Google says it has no user agent of its own.
4. Check that the requests are genuine. Bing and Google both warn that user-agent strings are easy to fake, so compare addresses with the IP lists OpenAI and Perplexity publish, or use Bing's Verify Bingbot tool.
5. Repeat every month. A crawler that fetches robots.txt and your pages but never llms.txt is giving you an answer.

For logs in the common combined format, two commands do the counting:

# Requests for /llms.txt, counted by user agent
grep '"GET /llms.txt' access.log | awk -F'"' '{print $6}' | sort | uniq -c | sort -rn

# The same for robots.txt, for comparison
grep '"GET /robots.txt' access.log | awk -F'"' '{print $6}' | sort | uniq -c | sort -rn

Do not read too much into a single fetch: Google notes that it may crawl many kinds of files without treating them specially. Logs settle the opposite question better: if no AI crawler that identifies itself requested the file in 30 days, none of them read it.

How do you see ChatGPT and other AI referrals in Google Analytics 4?

In Google Analytics 4, open Reports, then Acquisition, then Traffic acquisition, and find the AI Assistant row, which Google defines as visits from sources like ChatGPT, Gemini, DeepSeek, Copilot or Grok, excluding AI Overviews and AI Mode.

An AI referral is a visit that starts when someone clicks a link in an AI assistant's answer. It counts clicks, not citations: many answers are read without a click, and no GA4 report shows whether an engine ever looked at your llms.txt.

Google's default channel group help explains the rule: when the referrer matches Google's list of AI assistants, Analytics sets the medium to ai-assistant and the session lands in the AI Assistant channel. Google's downloadable list of sources leaves the AI assistants out, so check the main hosts yourself:

1. Open the report. Sign in, choose Reports, then Acquisition, then Traffic acquisition. The default dimension is Session default channel group; find the AI Assistant row.
2. Split it by source. Click the plus sign next to the dimension drop-down and add Session source, so the row breaks into chatgpt.com, gemini.google.com and the rest.
3. Search for each host. Switch the primary dimension to Session source / medium and type each host into the search box above the table: chatgpt.com, perplexity.ai, copilot.microsoft.com, claude.ai and gemini.google.com.
4. See where they land. Use + Add filter with Session source set to one assistant, then add Page path and screen class as a secondary dimension.
5. Compare periods. Note the 28 days before any change, including a new llms.txt, and the 28 days after, then check your logs before crediting the file.

Channel definitions from Google's Analytics Help, read on 22 October 2026.

Clicks from AI Overviews and AI Mode count as Organic Search; for citations rather than clicks, use the Search Console and Bing Webmaster Tools reports covered in the checklist.

When should a business website skip llms.txt?

Skip llms.txt, or leave it to your platform's generator, when your goal is Google rankings or AI citations, when your site has a few dozen pages, or when nobody will keep the file current; add one when AI agents use your documentation.

Rankings and citations. Google says Google Search ignores the file, and no other answer engine documents reading it. An hour spent on indexing and crawler access, the first steps of the get-cited checklist, does more.

Small sites. On a site of 10 to 30 pages with clear navigation, the file mostly repeats your menu, and an agent that reads your home page finds the same links.

No owner. A summary that quotes last year's prices contradicts your pricing page. If nobody will add a line for each new page, as we do, a stale file is worse than none.

Platform files. If Wix or Yoast SEO already generates one, read it once, correct anything wrong and let it update. On Wix, editing the file stops automatic updates until you reset it to the default.

When to add one. Developer documentation, an API, a set of integrations or a large help center are the cases the proposal was written for: its author says coding agents now use llms.txt files reliably. A store or service business with detailed policies can also give agents a clean map of them.

How do you create an llms.txt file in an hour?

To create an llms.txt file, check whether your platform already generates one, write an H1 and a dated summary, list 10 to 30 key pages with a note each, publish it at /llms.txt and test it with an AI assistant.

1. Check your platform. Wix generates and maintains the file for upgraded sites with a custom domain, under SEO & GEO, then Tools and settings, then Go to llms.txt. Yoast SEO generates one for free on WordPress. Mintlify adds an llms-full.txt copy of the whole documentation site.
2. Write the H1 and the summary. Give your name, what you sell, to whom and where, and two or three key numbers with a date.
3. List the pages that answer customer questions. Pricing, delivery, returns, booking, contact and your most-read help articles, grouped under H2 headings with a note each; our guide to formatting docs for an AI chatbot shows how to make the linked pages quotable too.
4. Publish and load it. Save the file as UTF-8 plain text at /llms.txt, open it in a browser and confirm it loads without an error.
5. Test it. Paste its address into an AI assistant, ask three questions your customers ask, and fix whatever it gets wrong.
6. Keep it current. Add a line to your publishing checklist for every new page, and reread the summary whenever prices or policies change.

Anything you list is public, so leave out pages behind a login.

What to do next

• Work through the checklist for getting cited by ChatGPT and Perplexity first: indexing and crawler access decide citations, and llms.txt is its smallest item.
• If you publish llms.txt, set a reminder for 30 days later to count requests for it in your logs, by user agent, before crediting it with any change in AI referrals.
• Put the same key pages to work on your own site: PepoChat's website import reads your sitemap or crawls up to 200 pages and honours robots.txt, and you can create a free workspace with every feature on the free plan.