#robots.txt last update pnevius 10-13-05 #tell everyone User-agent: * #stay out of cgi-bin (nevermind there ain't nothing there) Disallow: /cgi-bin/ #stay out of photos-there is stuff there Disallow: /photos/ # drupal specific User-agent: * Crawl-Delay: 10 Disallow: /aggregator Disallow: /tracker Disallow: /comment/reply Disallow: /node/add Disallow: /user Disallow: /files Disallow: /search Disallow: /book/print Disallow: /admin Disallow: /cron.php Disallow: /xmlrpc.php Disallow: /database/ Disallow: /includes/ Disallow: /modules/ Disallow: /scripts/ Disallow: /themes/ Disallow: */add/