Does Your Website Need an LLMs.txt File? + How to Create One

6 Steps to Get AI to Recommend Your Brand

There’s been a lot of buzz around llms.txt.

But no major AI platform has confirmed that they use it.

Not yet, anyway.

And there’s no evidence that any major large language model (LLM) actually uses it when crawling.

So, why are some SEOs and site owners already adding it to their sites?

Because LLM traffic is projected to explode over the next few years.

Which means AI models could soon become your biggest traffic source.

Remember: robots.txt was once optional, too.

Today, it’s essential for managing search crawlers.

LLMs.txt could follow a similar path — though the evidence so far doesn’t guarantee it.

May 2026 update: Google Search Central now explicitly states that LLMs.txt files are not necessary for its AI search surfaces.

Here are two caveats worth knowing:

  1. This only applies to Google. Whether LLMs.txt affects ChatGPT, Perplexity, or other platforms remains an open question.
  2. When SEO expert Lily Ray asked Google’s John Mueller why Google’s own properties still use the file, he clarified: It’s for agent functionality, not search.

His distinction — ”discovery” (how search engines find pages) vs. “functionality” (how agents help users complete tasks) — explains why the contradiction isn’t quite what it looks.

In this guide, you’ll learn how llms.txt files work, the key pros and cons, and the exact steps to create one for your site.

You’ll also see different llms.txt examples from real sites.

First up: a quick explainer.

What Is LLMs.txt?

LLMs.txt is a plain-text file that tells AI models which pages to prioritize when crawling your site.

This proposed standard could make your content easier for AI systems to find, process, and cite.

Here’s how it works:

For example, here’s what Yoast SEO’s llms.txt file looks like:

Does LLMs.txt Replace Robots.txt?

Short answer: No.

They serve different purposes.

Robots.txt tells crawlers what they’re allowed to access on a site.

It uses directives like “Allow” and “Disallow” to control crawling behavior.

LLMs.txt suggests which pages AI models should prioritize.

It doesn’t control access — it just provides a curated list. And makes it easier for crawlers to understand your content.

For example, you might use robots.txt to block crawlers from your admin dashboard and checkout pages.

Then, use llms.txt to point AI systems toward your help docs, product pages, and pricing guide.

Here’s a full breakdown of the differences:

LLMs.txt Robots.txt
Purpose Provides a curated list of key pages that AI models may use for information and sources Sets rules for search engine crawlers on what to crawl and index
Target audience LLMs like ChatGPT, Gemini, Claude, Perplexity Traditional search engine bots (Googlebot, Bingbot, etc.)
Syntax Markdown-based; human-readable Plain text, specific directives
Enforcement Proposed standard; adherence is not confirmed by major LLMs Voluntary; considered standard practice and respected by major search engines
SEO/AI impact May influence AI-generated summaries, citations, and content creation Directly impacts search engine indexing and organic search rankings

Layout and Elements

So, what goes inside this file — and how should you structure it?

LLMs.txt should be created as a plain-text file and formatted with markdown.

Markdown uses simple symbols to structure content.

This includes:

This makes the file easy for both humans and AI tools to read.

You can see the main elements in this llms.txt example:

# Title
> Description goes here (optional)
Additional details go here (optional)
## Section
- [Link title](https://link_url): Optional details
## Optional
- [Link title](https://link_url)

Now that you know how to format the file, let’s break down each part:

To put all the pieces together, let’s look at some examples.

Here’s how BX3 Interactive structures its llms.txt file:

It features:

BX3 Interactive also includes target terms and specific CTAs for each URL.

If the file is ever widely adopted, this approach could shape how LLMs reference the brand — but that’s still a big if.

LLMs.txt files can also be more complex, depending on the site.

Like this example from the open-source platform Hugging Face:

It organizes hundreds of pages with nested headings to create a clear hierarchy.

But it goes well beyond URL lists and summaries.

It includes:

This way, AI systems would get direct access to Hugging Face’s most valuable documentation without needing to crawl every page.

This could reduce the risk of key details getting missed or buried.

Keep in mind that the ideal structure depends on the scope of your site. And the depth of information you want AI to understand.

Is LLMs.txt Worth It?

The jury is out.

It’s possible that an llms.txt file could boost your AI SEO efforts over time.

But that would require widespread adoption.

No major AI platform has officially supported the use of llms.txt yet.

And Google has now made its position official. Its guide to AI optimization explicitly listed LLMs.txt as not necessary for AI search visibility. Which is the clearest statement the company has made on the topic.

That said, the situation is more complicated than a simple “Google said no.”

The agent vs. search distinction matters here: Chrome Lighthouse is adding llms.txt validation specifically for agentic use cases, which suggests the file may still have a role — just not the one most SEOs are optimizing for.

The usage data adds more context. A 90-day server log study by OtterlyAI found just 84 out of 62,100 AI bot requests hit /llms.txt — 0.1%. The top 50 most-cited domains actually have lower llms.txt adoption than lower-ranked sites.

But big players like Hugging Face and Stripe already have llms.txt files on their sites.

Most notably, Anthropic, the company behind Claude, also has an llms.txt file on its website.

Bottom line?

Treat llms.txt as a low-risk experiment, not a guaranteed way to boost AI visibility.

Potential Benefits

Right now, the benefits are theoretical.

But if llms.txt catches on, you could benefit in multiple ways:

Key Limitations and Challenges

The skepticism around llms.txt is valid.

Here are the biggest concerns:

How to Create an LLMs.txt File in 5 Easy Steps

Creating an llms.txt file is pretty simple — even if you don’t have much technical experience.

One caveat: You may need a developer’s help to upload it.

Step 1: Pick Your High-Priority Pages

Start by selecting the pages you want AI systems to crawl first.

Think about the evergreen content that best represents what you do — your core product pages, high-value guides, FAQ sections, key policies, and pricing details.

For example, BX3 Interactive lists this web development service page first in its llms.txt file:

Step 2: Create Your File

Next, open any plain-text editor and create a new file called llms.txt.

Options include Notepad, TextEdit (on Mac), and Visual Studio Code.

Not comfortable with markdown formatting?

Ask your developer to handle it (if you have one).

Or let an LLM do the work — ChatGPT and Claude can generate these files instantly.

Step 3: Upload the File

Where your llms.txt file goes depends on what it covers.

You might need a developer’s help for this.

Step 4: Make Sure It Works

Open a new tab and type in https://yoursite.com/llms.txt.

If you see your file displayed correctly, you’re set.

Step 5: Keep It Fresh

Your llms.txt isn’t a set-it-and-forget-it file.

Schedule a review every few months: