# Note: Robots that access TWiki.org at a high rate are # automatically added to a blacklist and will not be able # to access content anymore. Download the TWiki package, # it contains all TWiki.org pages you need to get started # with your own TWiki. # Note: Robots that access TWiki.org at a high rate are # automatically added to a blacklist and will not be able # to access content anymore. Download the TWiki package, # it contains all TWiki.org pages you need to get started # with your own TWiki. User-agent: iecrawler User-agent: ie_crawler User-agent: e-SocietyRobot Disallow: / User-agent: * Crawl-delay: 120 Disallow: /twiki/bin/ Disallow: /edps/edps-digest.archive