Does llms.txt Actually Work? What Google and OpenAI Have Said in 2026

Somewhere between 5% and 10% of surveyed sites now publish an llms.txt file, a plain-text summary of the site meant to help AI models understand it. It takes ten minutes to write and feels like the kind of small, cheap thing worth doing "just in case." The trouble is what happens after you publish it: in most cases, nothing, because nothing is reading it.

Google's own AI search optimization guide, published in May 2026, settles this more plainly than most SEO debates ever get settled. It says Google Search doesn't use llms.txt files and that adding one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." That's not a secondhand rumor or a forum opinion. It's the company's own documentation.

Google and OpenAI have taken different approaches to documenting llms.txt. Their published guidance helps decide whether the file deserves your time and what to check first.

TL;DR

• Google's May 2026 AI search guide states outright that it ignores llms.txt files for both rankings and visibility, a position Google has repeated multiple times since mid-2025.
• OpenAI has never officially confirmed that ChatGPT reads llms.txt. Its public crawler documentation describes robots.txt-based controls and says nothing about llms.txt.
• robots.txt is the opposite story: it's a decades-old, universally documented standard, and every major AI crawler fetches and respects it.
• llms.txt isn't harmful, just unproven. Publishing one costs little and won't hurt you, but don't expect it to move a single metric.
• The file does have one legitimate home: some developer tools, documentation sites and coding assistants reference llms-full.txt style files, which is a narrower, different use case than AI search visibility.
• Spend the effort on what's confirmed instead: robots.txt rules for AI crawlers, pages that actually return 200 to them, and content structured to answer a question directly.

Table of Contents

1. What llms.txt is and why it spread
2. What Google has said
3. What OpenAI, Anthropic and Perplexity have said
4. Is llms.txt worth doing?
5. What to check for AI visibility
6. How Rank Tim checks crawler access
7. Frequently asked questions
8. Sources and further reading

What llms.txt is and why it spread

llms.txt is a proposed convention, not a standard: a Markdown file at the root of a site, listing links to the pages a site owner considers most important, with short descriptions. The idea, floated in 2024, was straightforward and appealing: robots.txt tells crawlers what they can access, so why not a similar file that tells language models what a site is actually about?

It's cheap to create, and some CMS plugins generate one automatically. Teams looking for a way to improve AI visibility can add the file in minutes. That ease of publishing says little about whether a search system reads it. Google says its search system ignores llms.txt; OpenAI has not documented support for it.

What Google has said

Google has addressed llms.txt several times over roughly a year. Its position has stayed consistent.

Google's language became more explicit over time, moving from a comment about the lack of evidence to documentation stating that Search ignores the file. It also says publishing one neither helps nor harms visibility or rankings, so an existing file does not need to be removed for SEO reasons.

What OpenAI, Anthropic and Perplexity have said

Google's position is unambiguous. The picture for other AI companies is murkier, mostly because they've said far less.

OpenAI has never issued an official statement confirming or denying that ChatGPT reads llms.txt files. Its public documentation of its crawlers, which covers GPTBot, OAI-SearchBot and ChatGPT-User in detail, describes robots.txt-based access controls and says nothing about llms.txt as a recognized convention. The closest thing to evidence either way is a single site owner's server log screenshot from mid-2025 showing OpenAI's crawler requesting his llms.txt file roughly every 15 minutes, which he read as a freshness check. That's one data point, not a confirmed platform behavior, and it's never been corroborated by OpenAI.

Anthropic, interestingly, is the one major lab that publishes its own llms-full.txt style documentation, alongside companies like ElevenLabs and Pinecone. But this is a different use case entirely: structured docs for developer tools and coding agents to reference, not a mechanism for a company's marketing site to get cited more often in AI search answers. Conflating the two is where a lot of the confusion starts.

No platform has confirmed that llms.txt improves AI search visibility. Google says it does not use the file. Developer documentation for coding tools is a separate use case, with different readers and a different purpose.

Is llms.txt worth doing?

Writing the file can take ten to twenty minutes, with little maintenance beyond keeping it current. There is no confirmed benefit for AI search visibility and no evidence that it changes citations in ChatGPT, Gemini or Perplexity. The bigger cost comes when a team spends time on the file while a crawler access problem remains unfixed.

A team could spend a sprint building an elaborate llms.txt file while a robots.txt rule silently blocks OAI-SearchBot. Fixing that rule has a documented effect on crawler access; adding more detail to llms.txt does not.

What to check for AI visibility

Three categories of work have real documentation, real crawler evidence, or both, behind them.

Crawler access through robots.txt. This is the inverse of llms.txt in every respect: a decades-old standard, documented by every search engine and every major AI lab, and verifiable in your own server logs. If OAI-SearchBot, ChatGPT-User or Google-Extended can't fetch your pages because of an overly broad disallow rule, nothing else on this list matters. Our guide to AI crawlers and robots.txt covers which bots to allow and why, and our guide to tracking ChatGPT visibility shows how to read crawler hits from your own logs rather than guessing.

Structured data, where it's actually relevant. Google's own guide is careful here too: structured data isn't required for generative AI search either, but unlike llms.txt, it is documented, machine-parseable, and does help clarify entities, pricing, availability and reviews across systems that crawl structured markup, not just Google's. The difference from llms.txt is that JSON-LD has an actual specification and real-world adoption across the web, not a single proposal with no confirmed reader. Our schema markup guide covers which types are worth the effort.

Content built to answer the question directly. No file replaces this. AI engines, whatever their retrieval mechanism, are ultimately selecting text that clearly answers a specific question with evidence, not text that's merely present. Our guide to optimizing a site for AI search covers the content and structure fundamentals that keep showing up across every platform's own guidance.

How Rank Tim checks crawler access

Rank Tim's AEO Audit checks the things that are actually documented to matter, not the ones that sound technical. It verifies that your robots.txt explicitly allows the crawlers that feed AI search, OAI-SearchBot, ChatGPT-User and Google-Extended among them, and confirms your key pages return a clean 200 rather than a block page or a redirect loop to each one.

Notice what's not treated as a pass-or-fail item: llms.txt presence shows up as informational only, because treating it as a scored check would imply it affects your result, and neither Google nor OpenAI has confirmed that it does. The audit sits alongside AI Visibility, which tracks your actual prompts and citations across ChatGPT, Google AI, Gemini and Perplexity, so you can see whether the things that are confirmed to matter are actually working, rather than trusting a checklist that happens to feel thorough.

Frequently asked questions

Should I delete my llms.txt file if I already have one?

No need. Google has said explicitly that having one causes no harm to your Search visibility or rankings. There's no evidence it hurts you anywhere else either. The only real cost is the time spent maintaining it, so keep it simple if you keep it at all.

Does Bing or Perplexity use llms.txt?

Neither has issued a public statement confirming it. The clearest, most repeated statements on the record come from Google, which has said multiple times across 2025 and 2026 that it doesn't use the file. No other major platform has offered comparable documentation either confirming or denying use.

Is llms.txt the same thing as llms-full.txt used by developer tools?

They share a naming convention but serve different purposes. The developer-tooling version is documentation meant for coding assistants and IDE integrations to reference, a narrower and more established use case. The marketing-site version, the one most guides recommend for "AI visibility," is the one neither Google nor OpenAI has confirmed reading.

What should I check first if I'm worried about AI crawler access?

Start with robots.txt, not llms.txt. Confirm it explicitly allows OAI-SearchBot, ChatGPT-User and Google-Extended if you want those crawlers accessing your site, then check your server logs to confirm they're actually fetching pages, not just technically permitted to.

Does adding structured data help the same way llms.txt doesn't?

It's a legitimate but separate comparison. Structured data has an actual specification, is parsed by Google and by AI systems that crawl the open web, and clarifies specific facts like pricing or availability. It isn't required for generative AI search either, per Google's own guide, but it's documented and demonstrably read, which llms.txt is not.

If llms.txt doesn't help, why do so many SEO tools recommend it?

Partly inertia from 2024, when the idea was new and untested either way, and partly because it's an easy deliverable: a tool or agency can point to a generated file as proof of "AI optimization" work done. That doesn't make the underlying claim about its effect on rankings or citations true.

Sources and further reading

• Google Search Central: Top ways to ensure your content performs well in AI experiences
• OpenAI: Overview of OpenAI crawlers
• Search Engine Roundtable: Google Does Not Endorse LLMs.txt Files
• Search Engine Roundtable: Google: LLMS.txt Files Won't Help Or Hurt Your Search Rankings
• Search Engine Roundtable: Google: No AI System Currently Uses LLMs.txt