About Better Robots.txt

Better Robots.txt is a WordPress plugin for generating and managing robots.txt with a guided setup flow.

What the Current Version Focuses On

• Search-engine visibility presets
• AI training and AI search governance
• SEO crawler blocking
• Bad-bot protection
• Archive blocking on Premium
• WooCommerce crawl cleanup
• Resource allow rules for CSS, JS, and images
• Social-media crawler blocking
• ads.txt and app-ads.txt allow rules
• Optional llms.txt
• Custom rules, crawl delay, and output consolidation

How the Current UI Is Organized

• One Mode Selection screen
• 14 numbered steps
• A final preview inside Step 14 - Review & Save

Documentation Scope

This documentation is written for plugin users and follows the live setup flow shown in the plugin settings screen.