Subscription Confirmed

Your subscription to our list has been confirmed.

Thank you 

By the way, what is your Robots.txt ? You can enter a website below, click submit and it will detect if the site has a robots.txt file and display what the file says (it shows results here on this page).

Better Robots.txt The simplest and easiest way to fully benefit from your website.

By optimizing your robots.txt the right way, you can tell search engine bots to spend their crawl budgets wisely. And that’s what makes the robots.txt file so useful in an SEO context.

Test it or get it, but do something ! Don't take the chance of having useless content on your website.

FREEMIUM

$ 0

Available for download freely on WordPress
Allow to add indexing instructions to 16 crawlers
Easy to setup of crawl-delay

PREMIUM

$ 39

Only available on Better-Robots.com
Allow to add indexing instructions to 16 crawlers
Easy to setup crawl-delay
Verify your sitemap index from Yoast plugin
Add the sitemap URL to your Robots.txt
Block scrapers (Bad bots)

BOOST YOUR SEARCH ENGINE RANKING NOW‎!

Boost your content with your sitemap ! Make sure your pages, articles and products, even the latests, are taken into consideration by search engines !

Better Robots.txt plugin was made to work with Yoast SEO plugin (probably the best SEO Plugin for WordPress websites). It will detect if you are currently using Yoast SEO and if the sitemap feature is activated.

If yes, then it will add automatically an instruction in the Robots.txt file asking bots/crawlers to read the sitemap and check if you have made recent changes in your website (so that search engines can index the new content).

If you want to add your own sitemap (or if you are using another SEO plugin), then you just have to copy-paste the Sitemap URL and Better Robots.txt will add to your Robots.txt file.

From Better Robots.txt PRO plugin

Better Robots.txt

From Better Robots.txt PRO plugin

Boost your Website indexing Get your website in most popular search engine index !

We have identified a list of the most popular search engines (which will be updated frequently). Each time you select a bot, Better Robots.txt plugin add an instruction in your robots.txt file. Like this (if you add selected “GoogleBot”):

Better Robots.txt

This is a simple instruction asking Google to read and index your website.

We wanted it simple and we did it !

Protect your data and content Block bad bots from scraping your website and commercializing your information.

Better Robots.txt plugin offers an opportunity to block most recent detected bad bots from crawling and scraping your data.

When it comes to things crawling your site, there are good bots and bad bots. Good bots, like Google bot, crawl your site to index it for search engines. Others crawl your site for more nefarious reasons such as stripping out your content (text, price, …) for republishing, downloading whole archives of your site or extracting your images. Some bots was even reported to pull down entire websites as a result of a heavy broadband…

Better Robots.txt plugin is currently protecting your website against spiders/scrapers identified as bad bots by Distil Networks. We will provide with time (and upgrades) a more complete and detailed list of bad bots blocked by Better Robots.com.

Better Robots.txt

From Better Robots.txt PRO plugin

Great support to optimize your Robots.txt We will help you fix any WordPress indexing issues with our plugin.

Better Robots.txt
[Total: 0    Average: 0/5]