[Buildbot-devel] Few buildbot improvements

[Marcus Lindblom]

On 2010-04-10 22:24, Dustin J. Mitchell wrote:

>> b) Add some UI improvements: 1) Build's progressbar 2) Build's per-step time
>> chart. Without diving into details of why/what/how the progressbar/charts
>> could be implemented, first of all, I'd like to ask what's the project's
>> attitude towards embedding any JavaScript library(jquery-ui, raphaeljs.com,
>> or any other...) into the front-end? I know that's an important
>> ethical-philosophical issue to choose between usability/sleek look and
>> purism, but as some of the python's world
>> (namely http://djangoadvent.com/1.2/jquery-admin/) already sold their souls
>> for usability/sleek look, maybe that's not that evil thing to do after all?
>> :)
>
> The UI additions sound great!  Marcus was the prime force behind the
> web frontend at this point, so I will defer to his opinion on adding
> JS.  I have no objections.

jQuery rocks! So I'm perfectly ok with using that. (It could add a lot 
of neat features, such as more neatly settable refreshes, etc). I 
planned to do some JS with jquery earlier (to auto-expand 
revisions/usernames), but my HMI-fu wasn't up to it.

One thing we should do here to reduce the load of buildbot is to add the 
option to use google's jquery-code instead of always serving the local 
one. (At least until we can set HTTP-expires on the static data.)
Just (yet) another option to WebStatus. :)

This will reduce the load on public/busy buildbots quite a lot I believe.

> I do want to be careful that these additional UI features do not
> drastically increase the load on the buildmaster.  All web rendering
> is essentially done synchronously, so lots of HTTP requests can starve
> the buildmaster's attention to slaves.  Chris's work on slave-side log
> buffering reduces the impact of this problem somewhat, but not
> entirely.

I believe it can be written to reduce server-side load as much as 
possible, by using as little polling as possible.

I'd love to see some Comet[1]-stuff (which is the common name for server 
-> client HTTP push tech) for these things, but that probably requires 
Orbited[2] or something. (Unless there's something neat for twisted, not 
sure..)

My BuildbotIcon does a lot of polling too, and our developer-intranet 
page has 5 buildmasters which are polled every 10-15 secs (to show on 
the public screen). This polling adds up time.

Adding caching of HTML inside buildbot would've been cool, but I'm not 
too familiar with buildbot's internals and good cache invalidation 
strategies (which is Hard (tm) see quote[3]).

Cheers,
/Marcus

[1]: http://en.wikipedia.org/wiki/Comet_(programming)
[2]: http://orbited.org/
[3]: 
http://stackoverflow.com/questions/1188587/cache-invalidation-is-there-a-general-solution