What Better Robots /check does and does not do

Better Robots /check is a crawler-governance audit. It is designed to answer a precise question:

Does this public website express a coherent crawler and AI-use posture through robots.txt, AI crawler rules, machine-readable guidance, and governance signals?

It is not a general SEO audit. It is not a visibility tracker. It is not a firewall. It is not the WordPress plugin runtime.

Run the free auditPlugin vs /checkDeclaration vs blocking

What /check audits

/check evaluates public signals that are directly related to crawl governance and AI-use posture:

• robots.txt presence, availability, syntax, sitemap declarations and common crawl risks;
• search crawler access for Googlebot, Bingbot and other search baselines;
• AI crawler posture across training, AI search and user-triggered fetch categories;
• URL × bot access using robots.txt matching, so a specific path can be tested against selected crawlers;
• machine-readable guidance signals, including llms.txt, AI policy references, ai-manifest.json and .well-known governance pointers;
• WordPress crawl hygiene when a WordPress stack is detected;
• selected intent-profile fit, such as “AI search open, training restricted”.

The audit is deliberately narrow. Its strength is not breadth. Its strength is precision around crawler governance.

What /check generates

Depending on the detected stack and available public signals, /check may generate several artifacts:

Output	Purpose	Important limit
Audit report	Human-readable diagnosis and priorities	A report is not proof of crawler obedience
URL × bot matrix	Shows whether selected bots can access selected URLs according to robots.txt matching	It does not prove what a bot actually did in logs
Better Robots config JSON	Recommendation JSON for future WordPress import workflows	It becomes runtime state only after plugin import, preview and apply
Markdown remediation report	Technical handoff for a developer or agent	It is a plan, not an automatic server change
llms.txt draft	English machine-readable source map generated from public signals	It must be reviewed before publication

What /check helps fix on WordPress

/check is related to the Better Robots.txt WordPress plugin, but it is not the plugin itself.

The intended workflow is:

1. audit a public domain with /check;
2. select or confirm an intent profile;
3. generate recommendation artifacts;
4. import or apply supported recommendations through Better Robots.txt PRO when the site runs WordPress;
5. re-audit to verify the resulting public posture.

That loop is the product direction. The audit is the external diagnostic layer. The plugin is the WordPress application layer.

What /check does not guarantee

/check does not guarantee any of the following:

• crawler obedience;
• legal compliance;
• universal AI training exclusion;
• AI citation;
• AI ranking;
• traffic;
• downstream AI visibility;
• firewall-level blocking;
• server-side enforcement;
• complete SEO health;
• real model retrieval or brand mention outcomes.

robots.txt and related governance files are public declarations. They are valuable because they reduce ambiguity and guide cooperative systems. They do not replace WAF rules, bot verification, server controls, legal terms or log analysis.

When to use another tool

Use Google Search Console when the question is how Google sees a verified property.

Use a URL-level robots.txt validator when the question is only whether one URL is blocked by one user-agent.

Use Screaming Frog or a crawler when the question is a large-scale technical crawl.

Use a WAF or server rules when the goal is technical blocking.

Use an AI visibility tracker when the goal is measuring real mentions, citations or share of voice in model answers.

Use Lighthouse Agentic Browsing when the question is whether a page is operable by browser agents through accessibility, visual stability, WebMCP-like signals or interactive affordances.

The short version

Better Robots /check does not try to be everything.

It audits whether a site’s crawler and AI-use posture is explicit, coherent, machine-readable and actionable.