[Buildbot-devel] dissuading robots

[Justin Mason]

hi folks --

We're now seeing a fair bit of this:

2006/02/26 23:02 GMT [HTTPChannel,2565,66.249.65.71] 66.249.65.71 - - [26/Feb/2006:23:02:03 +0000] "GET /changes/375 HTTP/1.1" 500 113819 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
2006/02/26 23:02 GMT [HTTPChannel,2566,66.151.181.4] 66.151.181.4 - - [26/Feb/2006:23:02:25 +0000] "GET /mc-slower/builds/14/configure/0/text HTTP/1.1" 200 25586 "http://buildbot.spamassassin.org:8011/mc-slower/builds/14/configure/0" "FAST Enterprise Crawler agent 007(helpdesk at fastsearch.com / www.fastsearch.com)"
2006/02/26 23:03 GMT [HTTPChannel,2567,66.151.181.4] 66.151.181.4 - - [26/Feb/2006:23:03:46 +0000] "GET /mc-slower/builds/14/svn/0/text HTTP/1.1" 200 254 "http://buildbot.spamassassin.org:8011/mc-slower/builds/14/svn/0" "FAST Enterprise Crawler agent 007(helpdesk at fastsearch.com / www.fastsearch.com)"

etc. -- in other words robots hitting our Buildbots.  I'd prefer if they
didn't.

Is there any way to get a "/robots.txt" onto those HTTP interfaces?

--j.