#----------------------------------------------------------------------------- # robots.txt #----------------------------------------------------------------------------- # This file is under RCS control. # Use 'co -l robots.txt' to reserve it for modification. # Use 'ci -u robots.txt' to replace it when you are finished. # Use 'rcsdiff robots.txt' to see differences between the working version # and the checked-in version. # Use 'rlog robots.txt' to get a summary of changes that have been made. # # $Id: robots.txt,v 1.2 2015/01/27 03:58:27 root Exp $ #----------------------------------------------------------------------------- # This file only does "polite request" kind of blocking. # You can enforce the block (for impolite robots) with # .htaccess in the directory tree you want to block, or # in the directive of the web server configuration. # User-Agent: * Disallow: / # whole site - you may not want this Disallow: /cgi-bin/ # for twiki Disallow: /twiki/ # for twiki #