GlossaryGlossary
What is llms.txt?What is llms.txt?
llms.txt is a proposed standard (llmstxt.org) that publishes a machine-readable summary of your site for LLMs. It sits at /llms.txt and is similar to robots.txt or sitemap.xml. Major LLM crawlers check for it before consuming a site, and well-formed entries measurably improve AI citation rates.llms.txt is a proposed standard (llmstxt.org) that publishes a machine-readable summary of your site for LLMs. It sits at /llms.txt and is similar to robots.txt or sitemap.xml. Major LLM crawlers check for it before consuming a site, and well-formed entries measurably improve AI citation rates.
Glossary / What is llms.txt?
llms.txt is a Markdown file at the root of your domain (/llms.txt) that gives large language models a structured, machine-readable summary of your most important content. It was proposed by Jeremy Howard in 2024 and has been adopted as a soft standard by Answer.AI, Anthropic documentation sites, and a growing number of content publishers.
A typical llms.txt looks like:
``` # My Company
## About - [Company overview](https://example.com/about): one-paragraph summary in clean prose
## Products - [Product A](https://example.com/products/a): technical specs, use cases ```
The key principle is "Markdown for machines, not humans". Each link is followed by a clean prose summary of the page, written in a way LLMs can extract and quote directly. Unlike HTML pages, there is no navigation chrome, no JavaScript, no cookie banners — just signal.
If your site is missing llms.txt, GPTBot and ClaudeBot will still crawl your public HTML, but they will miss the curated summary of what your content means and what is most important. Empirically (ShortcutSistem scan data, July 2026), sites with llms.txt are cited 2-3x more often than equivalent sites without it.
ShortcutSistem's Omni Audit checks for /llms.txt presence and structure on every scan.
Related
| Term | |
|---|---|
| What is AI Visibility?AI Visibility is how easily ChatGPT, Claude, and Gemini can find, read, and recommend your website. It is the new SEO for the generative-search era. It depends on crawler access, structured data, entity consistency, and original content. | Read → |
| What is GPTBot?GPTBot is OpenAI's web crawler. It fetches public web pages to keep ChatGPT's training data and search index fresh. Site owners can block, allow, or partially allow GPTBot via robots.txt. If you block GPTBot, ChatGPT cannot cite your content in its generated answers. | Read → |
| How do LLMs decide which sources to cite?LLMs cite sources that are (1) accessible to their crawlers, (2) structured with schema.org, (3) consistent across the open web as an entity, and (4) original — they contain data or claims not found elsewhere. Backlinks and domain authority help but matter less than primary data. | Read → |
By Filbert Then · Last updated · Suggest a correction
Check your AI readinessCheck your AI readiness
Free. Structural signals only. About 30 seconds.Gratis. Hanya sinyal struktural. Sekitar 30 detik.