Robots.txt Generator
Build a robots.txt file to control which crawlers can access your site.
Loading tool…
How to use the Robots.txt Generator
- 1
Choose the default rule and add paths to allow or block.
- 2
Add your sitemap URL.
- 3
Download robots.txt and upload it to your site's root.
Frequently asked questions
Does robots.txt hide pages from Google?
It stops crawling, not indexing. Use a noindex meta tag to keep pages out of search results.