What Is llms.txt, and Does It Actually Improve AI Visibility?

llms.txt is a Markdown file placed at your domain root that curates high-value pages for AI crawlers. Proposed in September 2024 by Jeremy Howard of Answer.AI, it functions like a table of contents for large language models. Studies across 300,000 domains find no measurable lift in AI citations from the file, but it does real work as a routing layer for agentic developer tools. Worth 30 minutes to implement, not worth betting your GEO strategy on.

*By Josh Bernstein, Co-Founder | June 11, 2026*

The Origin: A Researcher's Solution to a Context Problem

In September 2024, Jeremy Howard of Answer.AI published a short proposal at answer.ai that laid out a practical problem: AI language models increasingly rely on website content, but most of a webpage's HTML is menus, tracking scripts, ads, and repetitive navigation. All of that structure consumes valuable context-window tokens without adding signal.

Howard's proposed fix was a plain text file at the domain root, formatted in Markdown, that hands an LLM a clean, curated map of the site's most important resources. He drew the analogy to robots.txt, a 1990s convention that web crawlers adopted as a de facto standard. The name of the file: llms.txt.

The proposal spread quickly through developer communities. By November 2024, documentation platform Mintlify rolled out automatic llms.txt generation across all its hosted docs sites. Almost overnight, high-profile developer properties including Anthropic, Cursor, and Hugging Face began serving the file. The problem is that broad adoption of a proposed standard is not the same as major AI platforms actually reading it.

What the File Contains (and What It Doesn't)

An llms.txt file lives at https://yourdomain.com/llms.txt. The spec is intentionally minimal. A valid file needs only three things:

• An H1 heading with the project or site name
• A short blockquote summary (one or two sentences)
• H2 sections grouping related content, each containing Markdown links with one-line descriptions

The spec also defines two optional companion files. llms-full.txt concatenates the full text of every linked page into one document, pre-loading context for any LLM that fetches it. llms-ctx.txt is a pre-expanded version for tools that want structured context without making multiple fetches.

A simple example for a SaaS company might look like this:

```

Acme Analytics tracks AI-search visibility for B2B SaaS brands.

Core Docs

• Getting Started: Install the tracking snippet and configure your first prompts.
• Citation Reports: How to read share-of-voice and citation-rate metrics.

Key Pages

• Pricing: Plan comparison and enterprise options.
• Case Studies: Customer results by vertical.

```

The format is so lightweight that most teams can ship one in under an hour, and several CMS plugins (Yoast, RankMath, and multiple dedicated WordPress plugins) now auto-generate the file from a sitemap.

Who Actually Uses llms.txt: The Honest Answer

This is where the practical picture gets complicated. "Who has published a file" and "who reads a file" are two very different questions.

Sites that have published llms.txt: Anthropic, Hugging Face, Vercel, Zapier, Cal.com, Perplexity, and thousands of developer documentation sites hosted on Mintlify. As of mid-2025, community directories listed over 784 implementations, though NerdyData's broader web scan found only 951 domains with the file as of July 2025.

AI platforms that have confirmed they use it: None. Google's John Mueller stated plainly that "no AI system currently uses llms.txt." OpenAI recommends using robots.txt for crawler control. Neither Anthropic, Google, Meta, nor Mistral has publicly committed to using the file as an input to their production answer or citation surfaces.

The most detailed empirical test was run by Semrush in collaboration with Search Engine Land. Their llms.txt file received zero visits from GPTBot, GoogleExtended, PerplexityBot, or ClaudeBot between mid-August and late October 2025. Crawlers simply did not fetch it.

The one category where llms.txt does demonstrably work is agentic developer tooling. IDE agents including Cursor, Windsurf, GitHub Copilot, Claude Code, Cline, and Aider routinely fetch llms.txt to load documentation context efficiently. For software companies whose buyers live inside these tools, that is a real and growing distribution surface. For a B2B SaaS marketing team trying to influence what ChatGPT says about their product, it is not.

The Data on AI Citation Impact

The most rigorous test to date was conducted by SE Ranking on a dataset of nearly 300,000 domains. Their XGBoost model, trained on AI citation data, improved its predictive accuracy when the llms.txt variable was removed from the feature set. The file was adding noise, not signal.

That finding is consistent with what we see at Promptive when tracking citation patterns across ChatGPT, Perplexity, Claude, and Gemini for client brands. The 2026 AI citation data shows that the factors predicting citation are consistent: external mention frequency, domain authority of citing sources, content that directly answers specific queries, and structured signals that help models parse what a brand does. A file telling a crawler "here are my important pages" does not move any of those levers.

The Princeton GEO study (KDD 2024, Princeton and Georgia Tech) put numbers to what actually does work: adding statistics to content improved citation rates by 41%, including quotes lifted performance by 28%, and citing authoritative sources produced up to a 115% improvement for lower-ranked pages (arxiv.org/abs/2311.09735). None of those gains come from a file at the domain root. They come from what is on the page.

llms.txt vs. The Signals That Actually Move Citations

The table below compares llms.txt to the technical and content signals with demonstrated impact on AI citation rates in 2026.

Schema markup is worth calling out specifically. Roughly 71% of pages cited by ChatGPT include structured data, compared to well under 50% for the broader web. FAQ schema no longer generates rich results in Google SERPs (Google removed FAQ rich results on May 7, 2026), and HowTo schema lost rich results back in 2023, but both types remain valuable signals that help AI engines parse content structure and extract quotable passages. For a deeper look at what earns citations across different platforms, see which sources AI engines cite most.

*Positioning of five core GEO signals by implementation cost and demonstrated AI citation impact. llms.txt sits in the low-cost, low-impact quadrant as of mid-2026.*

The One Context Where It Is Worth Shipping

If your audience includes developers, the math changes. Cursor has become the dominant environment for professional software engineering work in 2026, and agents running inside it and similar tools (Windsurf, GitHub Copilot, Cline, Claude Code) actively fetch llms.txt when they need to load documentation or understand what an API does. This is a B2A (business-to-agent) use case, not a B2C or B2B search use case.

For a developer tools company, a documentation platform, or any SaaS whose trial onboarding involves developers reading docs inside an AI IDE, an llms-full.txt file that includes your key reference pages is a real asset. It reduces the friction of agents loading your context, which can influence in-tool recommendations. Understanding how AI engines build answers from retrieved content helps clarify why this matters for the retrieval step specifically.

For a brand-level AI visibility program targeting ChatGPT, Gemini, or Perplexity citations in consumer or B2B contexts, this is a secondary concern. Run a GEO audit first to see where your citation gaps actually are before optimizing for a surface that may not affect your primary channels.

How to Build and Deploy llms.txt in 30 Minutes

If you decide the implementation cost justifies the optionality (and it usually does, given the 30-minute ceiling), the workflow is:

1. Create a file at /llms.txt on your root domain, not a subdirectory.
2. Start with an H1 containing your brand or product name.
3. Add a one-to-two sentence blockquote describing what you do and who you serve.
4. Group your most important pages into H2 sections: core product pages, key documentation, high-authority editorial content, and any original data studies.
5. Write a one-line description for each linked URL, focused on what a model would learn from that page.
6. Optionally, generate an llms-full.txt by concatenating the cleaned full text of each linked page. Tools like the llms.txt WordPress plugin, Yoast, and several static-site generators do this automatically.
7. Submit the URL in your robots.txt as an additional reference (not required, but tidy).
8. Verify the file is publicly accessible and returns a 200 status.

Do not put proprietary pricing logic, internal tooling, or content you would not want indexed in your llms-full.txt. The full-text variant is publicly readable.

What to Prioritize Instead

The sources that AI engines actually cite tell a clear story. Reddit accounts for roughly 40% of AI citations across models. Wikipedia covers about 13% of ChatGPT citations. High-authority editorial sources, research studies, and brand-owned pages with original data fill the rest.

The implication is concrete: a Wikipedia page about your company, a well-sourced original research report that earns third-party links, and a press hit in a tier-one publication will each do more for your AI citation rate than any technical file in your domain root. The Princeton study found that citing authoritative sources within your content improved citation rates by up to 115% for lower-authority pages. That is a structural content change, not a file deployment.

For brands worried about AI engines citing outdated information, the solution is the same: freshen the content itself, earn fresh external citations, and ensure AI crawlers are not blocked in your robots.txt. The file format carrying the content matters far less than the content's freshness and the authority of the sources referencing it.

Frequently Asked Questions

Does llms.txt improve my ranking in ChatGPT or Perplexity results?

No credible data shows it does. An SE Ranking study analyzing nearly 300,000 domains found that llms.txt was not a predictive variable for AI citations, and the model improved when the variable was removed. OpenAI, Google, and Anthropic have not confirmed they use the file as a citation signal. Implement it as a low-cost hedge, not as a primary GEO tactic.

What is the difference between llms.txt and robots.txt?

Robots.txt controls crawler access by allowing or blocking specific bots from specific URL paths. llms.txt does not control access; it curates context. It is a positive signal telling AI tools which pages are most important and what they contain, formatted in Markdown for easy LLM parsing rather than in robots.txt directive syntax.

Which AI tools actually read llms.txt today?

IDE agents including Cursor, Windsurf, GitHub Copilot, Claude Code, Cline, and Aider actively fetch llms.txt to load documentation context. Major answer engines including ChatGPT, Gemini, and Perplexity have not confirmed they use the file when composing responses or choosing citations.

Is llms.txt the same as an AI sitemap?

The terms are sometimes used interchangeably, but they are distinct. A sitemap.xml lists all indexable URLs for traditional search engine crawling. llms.txt is a curated shortlist, written in Markdown with human-readable descriptions, intended to give AI models the highest-signal pages rather than an exhaustive index.

Should I include my entire site in llms-full.txt?

No. Include only the pages you would want an AI agent or LLM to read as representative of your expertise and offering. Core documentation, key product pages, and high-authority editorial content are the right candidates. Thin pages, checkout flows, or any content with proprietary business logic should stay out.

How does llms.txt relate to schema markup for AI visibility?

They operate at different levels. Schema markup annotates individual pages with structured data that AI engines parse during content analysis; about 71% of ChatGPT-cited pages include structured data. llms.txt operates at the site level, pointing to important URLs. Schema has a stronger evidence base for AI visibility; llms.txt complements it rather than replacing it.

*The AI citation signal stack ranked by demonstrated impact. llms.txt forms the optional base layer; the highest-leverage work remains external authority and on-page content quality.*

The Honest Take for 2026

llms.txt is a well-designed proposal that solved the right problem for one specific use case: helping AI agents load clean documentation context without parsing noisy HTML. For developer tools companies, it is a genuine B2A optimization worth 30 minutes.

For the rest of AI visibility work, the evidence from mid-2026 is consistent. None of the major AI answer engines have confirmed they read the file. A 300,000-domain study found no citation lift. The Semrush crawl test found that major AI bots did not visit the file at all over a two-month period.

Google AI Overviews already reduce position-one organic CTR by 58% (Ahrefs, December 2025), so zero-click has become the default outcome for organic search. With 900 million weekly active ChatGPT users (OpenAI, February 2026), the stakes for earning AI citations are real. Spending those optimization hours on llms.txt rather than on original research, external mentions, and structured content that answers questions directly is a misallocation.

Ship the file. Set it and forget it. Then go do the work that actually moves citation rate.