[Buildbot-devel] Few buildbot improvements

[Tomas Čerkasas]

Hi folks!

as a part of my programming internship @ www.pov.lt , I've been given a task
to make a couple of improvements for the Buildbot. In brief:

a) Fix ETA calculations. The current text-output based estimation doesn't
quite work as expected: I guess "ETA 26s" switching to "ETA 4s" in 1s for a
primitive "sleep 30" ShellCommand step deserves being called working "not
quite as expected".

I'd tend to agree with
http://github.com/djmitche/buildbot/blob/master/buildbot/status/progress.py#L90
 comment
- the "non-time-uniformity of text output" could be the reason why
text-output based ETA (s) estimations might not be the best idea for each
use case.

Personally, I do fancy the simple time-based estimation. Though it doesn't
cope with the risks of slave machine being slower (possibly  because of
other processes using more hardware resources at the time) than during the
previous build, as long as I'm aware of this risk and I realize that for
this particular reason step takes longer than estimation, that's fine be me.
 This is the way estimates are done in hudson (hope that's not a swear word
in this mailing list:)  :
http://github.com/kohsuke/hudson/blob/master/core/src/main/java/hudson/model/Executor.java#L292and
http://github.com/kohsuke/hudson/blob/master/core/src/main/java/hudson/model/AbstractProject.java#L973.

The same functionality is achieved by commenting line 107 & 108 in
buildbot.status.progress.py (
http://github.com/djmitche/buildbot/blob/master/buildbot/status/progress.py#L107)
and forcing buildbot to always use the time-based ETA estimation, but that's
just a hack, not a solution.

Current scheme "If there are any metrics, use text-output based estimation,
otherwise, fall back to time-based estimation" is not transparent at all.
Some users might prefer the current "mix", while others(including myself)
would prefer to stick with time-only estimations, while even others probably
would prefer what is described by Brian in
http://buildbot.net/trac/ticket/540 . Some more great ways to calculate ETA
might be discovered later on... Therefore I think there is no one "right"
way to make ETA calculations, so maybe It'd be a good idea to introduce some
scheme for letting Buildbot users to choose which type of estimation method
they prefer and Buildbot Developers to add new ways of
calculating/estimating ETA easily.

Configuration: The first though crossing my mind on how to implement this
scheme is an optional setting in master.cfg, allowing user to choose default
ETA calculation/estimation setting. Then probably an optional step's
parameter, allowing to override the default setting.

Code-organization: should the different estimation/calculation methods be
all hardcoded into the buildbot.status.progress.StepProgress'es remaining()
method or maybe it's a better idea to normalize it to some external
class/interface?

I guess we'll be able to discuss it in more detail upon you expressing the
opinion if that's worth doing at all :)

+a minor fix to change the formatting of ETA(s) (from "ETA 732 seconds" to
"ETA 12 min. 12 s ") to make it more human-readable.

b) Add some UI improvements: 1) Build's progressbar 2) Build's per-step time
chart. Without diving into details of why/what/how the progressbar/charts
could be implemented, first of all, I'd like to ask what's the project's
attitude towards embedding any JavaScript library(jquery-ui, raphaeljs.com,
or any other...) into the front-end? I know that's an important
ethical-philosophical issue to choose between usability/sleek look and
purism, but as some of the python's world (namely
http://djangoadvent.com/1.2/jquery-admin/) already sold their souls for
usability/sleek look, maybe that's not that evil thing to do after all? :)

Upon receiving the feedback on the issues described above, if any of them
would seem worth attention, I would be more than happy to fill in the bug
report(s) and do my best to provide the patches...

Thanks,
to
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20100410/8dd887a9/attachment.html>