Why did we create Better Robots.txt and what difference does it make?

Blog