web-seo Tool
robots.txt Generator
Create standardized robots.txt directive files to control search engine indexing behavior.
robots.txt Generator & Directives Editor
Configure search engine bot access, allow/disallow paths, crawl delays, AI scraper blocks, and XML Sitemap locations.
Including the sitemap directive helps Googlebot and Bingbot index your site maps automatically.
User-Agent Directive Blocks (1)
#Block 1
Disallow (Blocked Paths)
/admin//private//api/
Allow (Explicit Exceptions)
/
Generated robots.txt Output
How robots.txt Generator Works
Generates Robots Exclusion Standard directive syntax for Googlebot and web crawlers.
Frequently Asked Questions
Where should robots.txt be uploaded?
Upload robots.txt directly to the root domain directory (e.g. example.com/robots.txt).