Understand LLMs.txt monitoring
Learn how TechDash treats the experimental llms.txt file and its changes.
Last checked 20 August 2026
TechDash requests /llms.txt during the daily site-resource run. llms.txt is an experimental convention for presenting selected site information to language-model tools. Its absence is not an SEO failure, an indexing directive, or a confirmed ranking factor.
The llms.txt row in Site Resources reports the latest HTTP status. The detailed lifecycle result appears in Sitewide Checks and can open an incident only after TechDash has stored a successful file.
What the daily request does
TechDash requests the website origin plus /llms.txt without following redirects. A 2xx response or 304 is treated as usable. It stores up to 100,000 characters of successful content, a body hash, the status, and the daily checked time.
The scheduler considers the daily resource set due 24 hours after its last completed daily check. It reviews due work every minute while monitoring is running. An offline computer, Wi-Fi sign-in page, paused website, unavailable license, closed app, or queue wait can move the actual check later.
Selecting Check now queues a forced resource run that includes llms.txt, but it does not bypass the single-worker Site resources queue.
Understand baseline behavior
The first successful response is stored as the baseline and produces no incident. If TechDash has never stored a successful hash and the current request is missing or unavailable, Sitewide Checks says that llms.txt has never been detected and remains Good.
After a baseline exists:
- LLMs.txt removed is Warning when the latest response is exactly
404. - LLMs.txt unavailable is Warning for any other unusable result, including a redirect,
4xx,5xx, timeout, or network failure. - LLMs.txt content changed is Advisory when a later successful body's hash differs from the previous successful hash.
These checks are never Critical by themselves. A failed request updates the visible status but keeps the previous successful hash and text. A later recovery can therefore be compared with the last good content.
The content-change incident includes previous and current text for a highlighted comparison. Any byte-level body change can alter the hash, including formatting, reordered links, or generated timestamps. Review the meaning of the change before treating it as a problem.
Investigate an incident
For removal or unavailability:
- Open the exact origin path, such as
https://www.example.com/llms.txt. - Compare the response with the incident's Expected and Current values.
- If it redirects, decide whether to serve it directly at
/llms.txt. TechDash deliberately does not follow the redirect. - Check CDN rules, deployment output, access controls, and content type.
- Restore the file only if your published AI-documentation policy still calls for it.
For a content change:
- Read the highlighted comparison.
- Confirm links still point to intended public, canonical resources.
- Remove stale, private, or accidentally generated content.
- Confirm the change with the team that owns documentation or AI access policy.
Do not create a file solely to make the card show 200 (OK). A maintained, accurate file may be useful; an abandoned or misleading file is not.
Know the boundaries
The built-in check monitors only /llms.txt. It does not validate a formal llms.txt schema, crawl every linked document, or prove that an AI service fetched or used the file. It is also separate from robots.txt AI crawler access. A site can publish llms.txt while blocking a crawler, or omit llms.txt while allowing crawler access.
Current Custom Check targets include robots.txt and sitemap XML, not llms.txt. Use the built-in lifecycle incident for this path.
After a correction, select Check now and watch both the llms.txt status row and Sitewide Checks. If the result remains stale, review the Site resources lane in Queue Status and verify that the website is not paused.