What is llms.txt?

llms.txt is a proposed standard created by Jeremy Howard (co-founder of fast.ai and Answer.AI) that provides a Markdown-formatted file at your website's root (/llms.txt) containing a concise, structured summary of your site's content specifically designed for consumption by large language models.

Think of it as the difference between robots.txt and llms.txt: robots.txt tells crawlers where they can go; llms.txt tells language models what your site is about. While robots.txt is a gatekeeper, llms.txt is a tour guide.

The Format

An llms.txt file uses simple Markdown with a specific structure:

# Score for AI

> AI readiness scanner that analyzes websites across 48 checks
> to measure compatibility with AI agents like ChatGPT, Claude,
> and Perplexity.

## Main Pages

- [Scanner](https://scoreforai.com/): Free AI readiness scan
- [Pricing](https://scoreforai.com/pricing): Full report for $29
- [Blog](https://scoreforai.com/blog): AI readiness guides

## Key Facts

- 4,100,000+ websites analyzed
- 48 AI readiness checks across 5 categories
- Results in 30 seconds, no signup required

The format starts with an H1 heading (your site name), followed by a blockquote description, then organized sections with links and key information. The entire file should be concise — typically under 500 lines.

There's Also llms-full.txt

The standard also proposes /llms-full.txt — a more comprehensive version that can include full page content, detailed documentation, or complete product descriptions. While llms.txt is the summary, llms-full.txt is the deep dive. Not all sites need both; start with llms.txt.

Who Supports It?

As of early 2026, llms.txt is still an emerging standard. It doesn't have formal support from OpenAI, Anthropic, or Google in the way that robots.txt does. However, it's gaining traction in the developer community:

• Several documentation platforms (Mintlify, ReadMe) have added automatic llms.txt generation
• Cursor and other AI coding tools check for llms.txt when analyzing project context
• Cloudflare added a one-click llms.txt generator for hosted sites
• The llms-txt.github.io registry lists hundreds of sites that have adopted it

While major AI chatbots don't specifically look for llms.txt during web browsing (they read the full page), the file can be useful for AI agents that need a quick overview of your site without crawling every page.

Should You Add It?

Short answer: yes, if it's easy for you. It takes 15-30 minutes to create and has no downside. Even if major AI agents don't specifically consume it today, it's a forward-looking investment that signals AI-friendliness and can help emerging tools understand your site faster.

Prioritize it if:

• You have documentation or developer-facing content
• Your site has many pages and complex navigation
• You want to control how AI systems summarize your business

Don't prioritize it over the fundamentals. If you haven't yet fixed your robots.txt, added structured data, or written substantial content, those should come first. Check our 10 steps to improve your AI readiness score for the priority order.

How It Fits Into AI Readiness

In the AI readiness framework, llms.txt falls under the Integration & Discovery category. It's one of several discovery mechanisms — alongside sitemaps, RSS feeds, agent.json, and MCP endpoints — that help AI systems find and understand your content.

Our AI readiness scanner checks for llms.txt as part of its 48-check analysis. Having it won't single-handedly transform your score, but combined with other discovery signals, it contributes to a well-rounded AI-ready website.

The broader trend is clear: the web is adding a machine-readable layer on top of the human-readable one. llms.txt, agent.json, structured data, and similar standards are all part of making the web more understandable to AI systems. The websites that adopt these standards early will be better positioned as AI agents become a primary interface for information discovery.