← All posts

llms-full.txt: the whole-site context file for AI agents

· Updated Nov 15, 2026 · 6 min read · By Mikhail Kuzmitskii

View as Markdown ↗

If you read nothing else: llms.txt is the menu, llms-full.txt is the whole meal in one plain-text file, and on a JavaScript-heavy site it’s the difference between an agent that reads it getting everything and getting nothing (for crawlers that skip the file, the fix is still server rendering).

I opened the llms.txt of a site I like, followed the first link the way a crawler would, and got a body that was one script tag waiting for a browser. The index was fine; the content behind it was invisible. That gap is exactly what llms-full.txt closes.

llms.txt is a curated index of a site; llms-full.txt is the site itself — the clean text of every page that matters, concatenated into one file. An AI agent (or any RAG crawler) that reads it gets the full context in a single fetch: no JavaScript to render, no navigation chrome to strip, no 40-request crawl to budget. It’s the companion file to the one-line llms.txt index, and in 2026 it’s probably one of the cheapest agent-readiness moves there is — though not a Google one: Google Search ignores it.

llms-full.txt: the whole-site context file for AI agents

llms.txt vs llms-full.txt

Axisllms.txtllms-full.txt
Purposecurated mapfull-text dump
Contentlinks + 1-line descriptionsthe actual page bodies
Sizesmall (a menu)large (the whole meal)
Read bya model deciding what to fetcha model reading everything at once
Analogythe table of contentsthe book, in plain text

They’re not rivals — they’re a pair, and arguably neither is much use alone. llms.txt tells a model what exists and links to per-page markdown; llms-full.txt lets a model skip the round-trips and ingest the lot in one go.

Why one big file beats a crawl

A crawler hitting rendered HTML typically fails in three ordinary ways, and I’ve watched all three happen on the same site! It can’t run the JavaScript, it wastes budget on nav/footer/cookie chrome, and it needs one request per page. Why fight three battles when a single clean-text file removes all three at once?

  • No render step. The content is already text. Client-rendered SPAs — the usual GEO black hole — become fully legible.
  • No chrome to strip. You ship bodies only, so the signal-to-noise ratio is ~100%.
  • One fetch, full context. A model comparing a pricing page to a competitor’s can hold the entire offering in context instead of guessing from one page.

What goes in it

Clean markdown bodies, in a stable order, each with enough front-matter for a model to attribute the passage:

  • The page title and canonical URL as a heading, so a citation can point back.
  • The body as markdown — headings, lists, tables preserved; scripts, nav, ads, and cookie banners removed.
  • A visible date (published / updated) — recency is a real GEO signal.
  • Nothing that isn’t already public. This file is quotable verbatim, and it will be.

Order the most important pages first (your money pages, your evergreen guides), because some crawlers truncate long files — and which page would you rather lose to truncation, the pricing page or an archived press release?

What to leave out

How to ship it

Three ways, cheapest first:

  1. Generate it from the content source. If your pages are markdown or a CMS with an export, concatenate the bodies with a title/URL/date header per page. This is what our GEO toolkit does automatically — it emits llms.txt, llms-full.txt, and a per-page .md alternate in one pass.
  2. Add the per-page markdown alternate too. Serve each page’s clean markdown at a predictable URL and advertise it with <link rel="alternate" type="text/markdown" href="…">. Then llms.txt links to those, and llms-full.txt inlines them.
  3. Deploy at the root and link it. Put the file at /llms-full.txt, reference it from /llms.txt, and keep both in the build so they never drift from the site.

Does it actually move the needle?

Honestly: Google Search ignores it, and it is not a documented ranking factor at any major engine — anyone selling it as one is selling something else. What it reliably does is remove failure modes — the render wall and the crawl-budget wall — for the agents and RAG tools that read it; for Google’s AI answers the fix is server rendering, not this file. For a client-rendered site, that gap is the difference between “invisible to RAG” and “fully readable.” Is it a growth lever? No. Is it a prerequisite? Only for the tools that read it — and Google isn’t one of them.

Where GEO crawlers lose your content (typical failure modes on a JS-heavy site)
JS render fails 100%
As clean markdown 0%

source: internal audits, 2026

The chart is the whole argument: a client-rendered page a crawler can’t execute is 100% invisible; the same page as a line in llms-full.txt is 100% readable to whatever fetches the file. Zero to everything, for the cost of a build step!

Checklist

  • /llms.txt exists (curated index, links to per-page markdown).
  • /llms-full.txt exists (full clean-text bodies, one file).
  • Each entry carries a title, canonical URL, and a visible date.
  • No HTML, nav, or boilerplate — bodies only.
  • Most important pages first; thin/duplicate pages excluded.
  • Both files regenerate on every build (never stale).
  • Per-page .md alternates advertised via rel="alternate".

In short

  • llms.txt is the index; llms-full.txt is the whole site as one clean-text file.
  • One fetch replaces a render step, chrome-stripping, and a multi-request crawl.
  • It’s plumbing: it removes reasons agents miss you, it doesn’t rank you (Google ignores it).
  • Bodies only, most-important-first, with title + URL + date per page.
  • Ship both, link them, and regenerate on every build so they never go stale.

It’s plumbing, and nobody compliments the plumbing — they only notice it the day the kitchen floods.

Sources

  • llmstxt.org — the original /llms.txt proposal (Jeremy Howard, 2024) and the llms-full.txt / expanded-context convention.
  • Aggarwal et al., “GEO”, KDD ‘24 — arXiv:2311.09735: citation depends on passage extractability, which clean text maximises.
  • Google, “Rendering on the Web” — why client-side JavaScript is a retrieval failure mode when the fetcher can’t execute it.

FAQ

What is the difference between llms.txt and llms-full.txt?

llms.txt is a short, curated index — a markdown list of your key pages with links and one-line descriptions. llms-full.txt is the expanded version: the actual clean-text body of those pages inlined into one file, so a model gets the full content in a single fetch instead of crawling each link.

Do AI engines actually read llms-full.txt?

It is an emerging convention (llmstxt.org), not a ranking signal. Google's May 2026 guide says Google Search ignores llms.txt-style files, and no other major engine has confirmed reading them; the real audience is agents, coding tools and RAG pipelines. It costs almost nothing to ship, it is trivially machine-readable, and it removes JavaScript-render and crawl-budget as failure modes for the tools that do use it.

Where do I put llms-full.txt?

At your site root: /llms-full.txt, served as text/plain or text/markdown, and linked from /llms.txt. Keep both current — a stale context file is worse than none.

Was this helpful?
Launch SEO/GEO with seo·matrix → or a request without Telegram →

Comments

  • …

Comments are moderated.