Robots.txt Generator

Generate robots.txt files for your website.

Pages/directories you want to block
Specific pages to allow within blocked directories

About This Tool

The robots.txt file tells search engine crawlers which pages or files they can or can't request from your site. Upload this file to your website's root directory.