Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

AllToolSet Robots.txt Generator makes a robots.txt file simple. Custom rules can be set up, and a ready-to-use file can be generated and uploaded rapidly.

This file, known as the robots. txt, tells search engine spiders which URLs on your site they may visit. In order to prevent your site from becoming overburdened with requests, this is utilized. However, it is not a means for keeping a web page out of Google's search results.

AllToolSet Robots.txt generator can be used by anyone, even if they aren't programmers. You just need to follow a few simple steps and you'll be done! You'll have a customized robots.txt file in less than a minute that you can use right now on your site. Regardless of your skill level, it's ideal for both novices and specialists.