Governance
This page explains the governance posture behind Better Robots.txt.
Why governance matters here
Better Robots.txt sits at the intersection of:
- search visibility
- crawler control
- AI-era usage preferences
- operator clarity
- final output review
That means the product needs to be honest about what it can and cannot do.
Declarative vs enforcement
Better Robots.txt publishes:
robots.txtrules- optional
llms.txt - machine-readable public files
- website guidance for people and machines
These are declarative surfaces. They express intent and policy.
They do not guarantee crawler compliance.
What the plugin is not
The plugin is not:
- a firewall
- a hard anti-scraping enforcement engine
- a replacement for server security
- a legal compliance engine