Understand AI crawler, WebMCP, and agent accessibility checks
Interpret experimental AI crawler, declarative tool, and control-label regressions.
Last checked 21 August 2026
These checks report narrow readiness and regression signals. A pass means the measured policy or markup is unchanged, absent, or within its baseline. It does not prove AI visibility, agent compatibility, accessibility, or SEO quality.
AI crawler access
AI crawler access removed is a Warning that matters because an accidental robots.txt change can remove permitted access for services used for training, retrieval, or citation. On every successful hourly robots.txt fetch, TechDash evaluates GPTBot, ClaudeBot, PerplexityBot, and Google-Extended separately. It applies explicit user-agent groups before *, and supports wildcard, end-anchor, longest-match, and equal-length Allow precedence.
The first successful policy is a true per-crawler baseline and passes as “baseline stored.” A later pass means none of the four crawlers changed from allowed at the prior successful policy to blocked at /; it can also mean a crawler remains blocked. A fail occurs only for a crawler whose prior policy allowed / and whose current policy blocks /. Evidence is the exact comma-separated names of newly blocked crawlers, not every currently blocked crawler.
If robots.txt is unavailable, evaluation is skipped. Any prior failure is preserved rather than falsely recovering. The evidence does not prove network enforcement or successful crawling: robots rules are voluntary, and a CDN or WAF can allow or block requests independently. Benign failures include an approved legal or content policy change. Review the robots.txt diff and bot controls, classify each crawler's intended role, and restore access only if the block was accidental. Verify recovery with a successful robots.txt fetch and Check & Verify; the next result must show no newly blocked crawler. Confirm network access separately in CDN logs.
WebMCP tools and validity
TechDash extracts declarative WebMCP attributes from forms in rendered mobile HTML. This experimental family matters because frontend changes can silently alter agent-facing tool contracts, but it does not observe imperative JavaScript registration or prove an agent can complete an action.
WebMCP first detected, WebMCP tool removed, and WebMCP tool changed are Advisory findings. The fingerprint contains normalized tool name, description, and sorted parameter names. A pass means the fingerprint is stable or no tools are present. First detection fails informationally when an empty prior page baseline gains tools. Removal fails when a previously non-empty set becomes empty. Change fails when both sets are non-empty and their fingerprints differ. WebMCP cohort coverage loss fails at finalization when at least two monitored pages changed from tools present to none; otherwise it passes. Evidence shows fingerprints or affected paths, but not executable behavior, parameter types, authentication, confirmations, or transaction safety.
WebMCP became invalid is Advisory. It fails only when a page with previously present, issue-free tools gains either a description without a name or a named tool without a description. A pass can mean valid, absent, already invalid, or no eligible prior tool baseline. Despite broader product documentation, the check does not currently detect required controls without names or missing parameter descriptions. Forms missing declarative WebMCP always passes as an opportunity: absence is not a failure.
Benign findings include intentional schema versioning, tool retirement, or adding WebMCP for the first time. Compare the rendered form and fingerprint with the release, restore attributes if accidental, and manually test names, parameters, validation, authentication, and side effects. After deployment, run Check & Verify and wait for finalization when checking cohort recovery.
Agent-centric accessibility
Agent accessibility regression is Advisory. It matters because unnamed controls can obstruct assistive technology and browser agents. After a baseline, it fails per page when unnamed rendered controls increase by at least three, or when named controls decrease and unnamed controls increase. A pass means those count thresholds were not crossed, including a first baseline; it does not mean the page conforms to WCAG.
Evidence contains previous and current named and unnamed counts plus sample selectors for current unnamed controls. Samples are diagnostic, not a complete accessibility tree or complete element inventory. The check does not test keyboard order, focus, contrast, live regions, dialog behavior, errors, or control usability. Benign changes include removal of genuinely interactive controls or a corrected extractor classification, but decorative elements should not be made interactive without purpose.
Restore visible labels, associated <label> elements, or accurate accessible names. Test the actual controls with keyboard and assistive technology. Run Check & Verify after the fix; recovery requires counts that no longer meet the regression trigger, while broader accessibility still needs dedicated testing.