Robots.txt Generator
Enter your sitemap URL and copy a ready robots.txt block. The file allows every user-agent on all paths and appends a Sitemap line so crawlers can find your index quickly.
Updated 2026-08-03 · seo
User-agent: *
Allow: /
Sitemap: https://example.com/sitemap-index.xml
About this tool
One Sitemap URL field drives the output. The generated text is always User-agent: * then Allow: / then a blank line and Sitemap: followed by your URL. Change the input and the preview updates immediately.
How to use
- Open the generator: Load the page so the Sitemap URL field is visible.
- Enter the sitemap URL: Paste your sitemap or sitemap-index URL (sample uses https://example.com/sitemap-index.xml).
- Copy the robots.txt: Select the result block with User-agent, Allow, and Sitemap lines.
- Deploy the file: Save the text as robots.txt at your site root and verify it loads over HTTPS.
Common use cases
- Bootstrap robots.txt for a new site that should be fully crawlable.
- Point search engines at sitemap-index.xml or sitemap.xml in one paste.
- Hand a starter file to hosting before you add finer Disallow rules later.
Tips
- Use a full absolute URL for the sitemap (https://…), not a relative path.
- This starter allows everything—add Disallow lines manually if you need private areas.
FAQ
What rules does the output include?
User-agent: * and Allow: / plus a Sitemap line with your URL.
Can I block paths with this UI?
Not in the form—the generator is a permissive starter; edit Disallow rules after copying.
Does the sitemap URL have to exist yet?
The tool only formats the file; hosting a real sitemap is still your responsibility.
Is the URL sent to a server?
The robots.txt string is built in your browser as you type.