[Buildbot-devel] html.Waterfall robots.txt support patch

[Tobi Vollebregt]

> Does anyone on the list actually want a robots.txt besides
> "User-agent: *\nDisallow: /\n"?  Is there any reason that shouldn't
> just be hard-coded into the server?  (That was my lazy solution.)

My reason to do it this way, was that I could just look at how it was done for 
the css and the favicon and copy that ;-)

Also, IMHO, it (almost) never hurts to provide more functionality then 
strictly necessary for 90% of the users. You never know: someone might want 
their home brewn log crawler to index the logs, but refuse other search 
engines. Or in the future buildbot is extended to show other pages which make 
more sense to index?

Also it's just more consistent with the other arguments: css and favicon are 
filenames too.

Anyway, for me, your lazy solution would have been good enough too, but this 
seemed easier to implement for me.

Tobi Vollebregt