# /robots.txt file for 21st Sentury Computing sites User-agent: * Disallow: /htsdata Disallow: /ftp Disallow: /cgi-bin Disallow: /logs Disallow: /mailbots Disallow: /stats