Basic Configuration
A safe default path
For most WordPress websites:
- choose Essential
- keep search engines allowed
- keep the final preview step in your workflow
- avoid changing advanced directives on day one
Recommended first decisions
Search engines
Allow major search engines unless you have a specific reason not to.
AI and LLM settings
If you publish original content and want a measured AI-aware setup, consider AI-First.
Social preview bots
Do not block them unless you know you no longer need social link previews.
WooCommerce cleanup
Use it if your shop generates many low-value dynamic URLs.
Then iterate
Better Robots.txt is designed so you can start safely and then move toward stricter or more detailed controls when the business case is clear.