[Buildbot-devel] [patch] Add support for robots.txt
Brad Hards
bradh at frogmouth.net
Wed Oct 26 10:24:49 UTC 2005
After looking at my logs for a long time, I'm sick of watching bots
(especially yahoo's slurp spider) wandering across the bot. The attached
patch (plus file) provides support for the robots.txt file.
Thoughts?
Brad
-------------- next part --------------
User-agent: *
Disallow: /
-------------- next part --------------
A non-text attachment was scrubbed...
Name: buildbot-robots-2005-10-26.patch
Type: text/x-diff
Size: 4852 bytes
Desc: not available
URL: <http://buildbot.net/pipermail/devel/attachments/20051026/7f77a04c/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://buildbot.net/pipermail/devel/attachments/20051026/7f77a04c/attachment-0001.bin>
More information about the devel
mailing list