Robots.txt Generator

Build robots.txt rules visually, apply presets, and download the file for your site.

Quick Presets

Rules

User-agentAllow / DisallowPath

Sitemap URL (optional)

Generated robots.txt

# robots.txt

Note: robots.txt is advisory only — well-behaved bots follow it, but malicious crawlers may ignore it. It is not a security mechanism.

How to Use

Click a Quick Preset to load a common configuration, or build rules from scratch.
Each rule row has a User-agent (which bot), an Allow/Disallow directive, and a Path.
Use * as the User-agent to target all crawlers.
Paths must start with /. Use / alone to match everything.
Add your Sitemap URL to help search engines discover your content.
Click Copy or Download to use the generated file. Place it at the root of your domain: https://yourdomain.com/robots.txt

About this Robots Txt Generator

Use this free robots.txt generator to create basic crawler rules for a website. It is useful for SEO setup, test sites, blocking sections, allowing search engines, and adding sitemap references. No signup is required, and the tool works on phones, tablets, and desktop browsers.