Build and test a robots.txt
Includes one-click blocking for GPTBot, ClaudeBot, Google-Extended, CCBot and the rest — kept separate from search crawlers, because confusing the two removes you from Google.
- Runs entirely in your browser
- Works offline
- No upload
- No sign-up
- No watermark
Loading the tool…
How it works
- List the paths to disallow, and any exceptions to allow.
- Optionally block the AI crawlers.
- Test a URL against the result — the tester uses the longest-match rule search engines actually apply.
Why nothing is uploaded
Every operation on this page is done by code running inside your browser tab, using the same engine that renders web pages. The file is read from disk into your tab’s memory, transformed there, and written back out as a download. It is never sent anywhere — not to us, not to a third party.
Verify it yourself
- Open your browser’s developer tools (F12) and select the Network tab.
- Load your file and run the tool.
- The only requests you will see are the tool’s own code and, once cached, none at all.
Frequently asked questions
Does blocking Google-Extended hurt my search ranking?
No. Google-Extended controls whether your pages are used for Gemini and model training; it has no effect on Google Search indexing or ranking. Googlebot is the one that matters for search, and blocking that removes you from results entirely.
Does robots.txt keep a page out of Google?
No — that is the most expensive misunderstanding in technical SEO. Disallow stops crawling, not indexing: a blocked URL can still appear in results, listed with no description, if other sites link to it. To keep a page out, allow crawling and use a noindex meta tag.
Does anything leave my browser?
No. There is no server behind this page doing the work and no API being called — the whole tool is JavaScript running in the tab. Load it once and it keeps working with the connection off.