Robots.txt Generator
Generate a robots.txt file using a visual form builder. Add rules, presets and sitemap links.
Default Behaviour
Presets
Add Rule
Options
robots.txt Preview
# robots.txt — generated by SimplyToolbox User-agent: * Allow: /
What the Robots.txt Generator does
A robots.txt file sits at the root of your domain and tells crawlers which parts of your site they may or may not visit. Getting it right matters: one stray Disallow can accidentally hide your whole site from Google, while a missing rule can let unwanted bots hammer your pages. This tool builds a correctly formatted robots.txt through a visual form, so you never have to remember the exact syntax of User-agent, Allow and Disallow lines.
It is aimed at site owners, developers and SEO practitioners who want a quick, valid file without typos. You choose a default behaviour, add specific rules, optionally set a crawl delay and a sitemap link, and the live preview shows the finished file ready to copy or download.
How to use it
Begin with Default Behaviour: tick Allow all bots to welcome every crawler, or Block all bots to keep everyone out (useful for staging sites). Then add granular rules in the Add Rule section by entering a user-agent such as * or Googlebot, choosing Allow or Disallow, and typing a path like /admin/. Each rule you add appears in a list you can remove individually.
For common scenarios, click a preset such as Block AI Scrapers, Allow Google Only, Block Bad Bots or WordPress Standard to drop in a ready-made rule set. Add a crawl-delay in seconds if you want to slow down aggressive bots, paste your sitemap URL so crawlers can find it, then click Copy to grab the text or Download to save a robots.txt file directly.
Tips and notes
Robots.txt is a request, not a hard lock. Well-behaved crawlers like Googlebot honour it, but malicious bots can ignore it, so never rely on it to protect private data; use authentication for that. Place the finished file at the root of your domain (for example https://example.com/robots.txt) for it to take effect.
Be careful with Disallow: / under User-agent: *, as it blocks your entire site from search engines. Always include your sitemap line to help crawlers discover your pages, and remember that the AI-scraper and bad-bot presets only deter crawlers that choose to respect the file.