robots.txt Builder
Build a robots.txt file by toggling common WordPress rules and adding your own paths and sitemaps. The file assembles live in your browser, ready to copy or download.
User-agent
WordPress core paths
Query strings and pagination
WooCommerce
Custom rules
Sitemaps
About the robots.txt Builder
The robots.txt Builder helps you create a robots.txt file for a WordPress site by toggling common rules instead of writing the syntax by hand. Start from a preset (standard WordPress, WooCommerce, strict, or allow all), then turn individual rules on or off, add your own disallow paths, and list one or more sitemap URLs. The file is assembled live so you can see the result as you change options, then copy it or download a ready-to-upload robots.txt. Everything runs in your browser, so nothing is sent to a server.
How it works
- Pick a preset to load a sensible starting set of rules, or start from scratch with allow all.
- Toggle the WordPress, query string, and WooCommerce rules you want, and add any custom disallow paths.
- Add one or more sitemap URLs so crawlers can find your XML sitemap.
- Copy the generated text or download it as robots.txt, then upload it to your site root.
Features
- Presets for standard WordPress, WordPress with WooCommerce, strict, and allow all.
- Common WordPress rules including disallow /wp-admin/ with an allow for /wp-admin/admin-ajax.php.
- Editable rules: user-agent, allow paths, disallow paths, crawl-delay, and custom disallow entries.
- Multiple sitemap URLs, including a Rank Math or Yoast sitemap index option.
- Live preview in a monospace box with one-click Copy and Download as robots.txt.
Frequently asked questions
Where do I put the robots.txt file?
Upload it to the root of your domain so it is served at https://yourdomain.com/robots.txt. On WordPress, place it in the same directory as wp-config.php, or use your SEO plugin's file editor if it manages robots.txt for you.
Why allow /wp-admin/admin-ajax.php while blocking /wp-admin/?
Many themes and plugins load front-end features through admin-ajax.php. Blocking the whole /wp-admin/ folder but allowing that one file lets crawlers fetch resources they need to render pages while still keeping the admin area out of search results.
Does blocking a path in robots.txt remove it from Google?
No. robots.txt asks crawlers not to fetch a path, but a blocked URL can still be listed if other pages link to it. To keep a page out of search results, use a noindex meta tag or header on a page that crawlers are allowed to read.
Should I disallow /wp-content/?
Be careful. Blocking /wp-content/ can stop crawlers from loading CSS, JavaScript, and images, which can hurt how Google renders and ranks your pages. This tool only offers blocking the plugins and themes source folders, not the uploads folder.
Is my data sent anywhere?
No. The file is built entirely in your browser from the options you select. Nothing you type, including sitemap URLs and custom paths, is uploaded or logged.