[Buildbot-devel] How to create sub-columns for each buildslaves under the same builders in waterfall

[Axel Hecht]

FYI, I have a waterfall display on top of my local status db that does the
following:

Finished builds get shown as a single box, in-progress builds get their
individual steps shown. This greatly reduces the amount of information you
might not really care about.

I also keep track of parallel builds and only open additional columns if
needed. Slightly buggy right now, though.

So, as one that has written a waterfall, yeah, beasty algorithms coming at
you there.

Axel

2010/1/27 Marcus Lindblom <macke at yar.nu>

> On 2010-01-27 04:56, Charles Lepple wrote:
> > On Jan 26, 2010, at 9:22 PM, Dustin J. Mitchell wrote:
> >
> >> I think your first question should be: is the waterfall the best view?
> >>
> >> For most purposes, either the console or one of the grids is the most
> >> useful display - they show the things developers are interested in -
> >> commits and builds - while omitting the things that developers don't
> >> care for - relative timing of various steps.
> >
> > As someone who used to run Buildbot on a 30+ minute build, the
> > waterfall is the most useful way to visualize *progress* on the build
> > (rather than a simple pass/fail status). We used the individual step
> > labels, as well as a custom status procedure that kept track of the
> > last directory that the build process descended into.
>
> I've been thinking to add some kind of progress to the grid or
> one_box_per_build result. Like two progress bar, one that works on time
> (estimation) and another that shows % of all steps completed.
>
> I'm thinking something like Hudson's bars, in the boxes for each
> builder: http://wiki.hudson-ci.org//download/attachments/753667/1.png
>
> (Hudson has a rather slick UI I must say...)
>
> > The build
> > process also kept running after errors ("make -k"; not my decision) so
> > this was the easiest way to see if a large chunk of code was being
> > skipped over due to failed dependencies.
>
> This is what we do as well, since we want to isolate errors as far as
> possible, then it's a good idea to build as much as possible. I haven't
> added a custom step that flags the build as failed as soon as errors
> appear in the log though. That is probably a good thing to have when
> running with -k (or similar.)
>
> > I would agree that the waterfall is not the best place to start
> > hacking if you are new to Buildbot and Python, but I also hope that
> > the waterfall doesn't start to fall by the wayside just because there
> > are a few more high-level summary views.
>
> Not at all. The waterfall has its place, especially when debugging locks
> or looking at detailed progress. But the code _is_ a big spaghetti pile,
> although it might be slightly easier to work with in 0.8. (F.ex, I
> didn't manage to extract all HTML from the python code there when
> converting to jinja. More work there is welcome, so that one is able to
> embellish that data.)
>
> Also, as Dustin said, views that access a lot of Buildbot's stored data
> are a bit slow, and the waterfall is one example. I suppose adding more
> features is nice, but we might need to address the performance if we do
> even more stuff here.  (I'm seeing response times of 0.5-2 secs here
> when I have three builders with a few hundred builds each. This is not
> ok for public buildbots.) Some form of caching of generated html would
> be nice to have here, so that the we only regenerate the boxes contents
> if they actually changed. That'd keep the Waterfall from digging deep
> into build-history every time.
>
> Actually, I'm not sure the grid & console views are good performers
> either...
>
> /Marcus
>
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the
> business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20100127/d0a0c913/attachment.html>