Robots.txt auto-checker No traffic on your website? Wondering why?
We may have the answer: 1/5 of WordPress websites are blocked because of a misconfigured robots.txt.
Test your website, it literally takes 2 seconds to get instant results.
Enter your website URL below (with http:// or https://), click SUBMIT and our tool will detect if you have a robots.txt file and display its content (it shows results here on this page).
*We don’t keep or track any of your information. This is just a passive tool.
What's your result ? The robots.txt file is a simple text file placed on your web server which tells webcrawlers (like Googlebot) if they should access a file or not.
- The robots.txt file controls how search engine spiders see and interact with your webpages;
- This file, and the bots they interact with, are fundamental parts of how search engines work;
- The first thing a search engine crawler looks at when it is visiting a page is the robots.txt file;
- The robots.txt is a source of SEO juice just waiting to be unlocked.
Have some trouble using our tool ? Send an email to [email protected] to be in touch with our team.
If you have this result:
It means that your website is blocking all Webcrawlers (indexing bot – search engines) from accessing & crawling your content and consequently generating ranking.
This rule: “Disallow /” asks all spiders to NOT crawl your website !