[Buildbot-devel] Merge requests troubleshooting

[Ludovic Chabant]

Actually, the more I look at the code and documentation, the more I think I
may be mistaken as to how the build request merging works -- and to some
extent, how the build prioritization works too.

Correct me if I'm wrong, but it looks like those 2 things (prioritizing and
merging) happen at the scheduling level, i.e. after a changesource polled
some changes and the schedulers declare what builds they should start,
those builds gets prioritized and merged. Then they get executed and
nothing else happens.

This is nice for the situation where you want to handle changes coming in
fast from your source control. However, my situation seems to be slightly
different: changes are coming in slowly enough that my changesource only
gets one or two changes every time, but my builders are so slow that they
start piling up after a few hours with dozen of them pending on my slaves.

What I want to do is this:
- Slaves are running only 1 or 2 builds each a the same time.
    - I tried using both slave locks or the `max_builds` property on a
slave, but this seems to be handled after prioritization/merging.
- Using either locks or `max_builds`, builders are put in the "pending"
state, waiting for an available lock or slave.
- More builds come in from the changesource
- >>> Those new builds get merged with any compatible pending builds. <<<
- Next time a slave or lock gets freed, the next pending build is started,
but by now it's a "bigger" build that will build more changes than when it
was originally created (while waiting, it got merged with incoming builds).

Is that possible at all with buildbot? Is there another way to achieve the
same goal? Is this stupid?

Thanks



--
   l u d o .
   . 8 0 17 80


On Tue, Jul 9, 2013 at 9:32 AM, Ludovic Chabant <ludovic at chabant.com> wrote:

> Hey everyone,
>
> I enabled the "merge requests" feature in buildbot but as far as I can
> tell, not much is getting merged, and my slaves are falling behind with
> many pending builds.
>
> How can I figure out what's going on? The only thing I can think of is
> create a custom merge function that implements the default behaviour, but
> with calls to "log" to print information out to the log... I don't know if
> there's a better way.
>
> Also, it may be relevant to note that most of the builds scheduled by a
> change are "empty" builds that only have "Trigger" steps to trigger various
> other builds (e.g. one build per platform and configuration). I wonder if
> buildbot thinks it doesn't need to merge that first build because it
> executes so quickly, and then all the triggered builds start clogging the
> master because it won't try to merge those?
> --
>    l u d o .
>    . 8 0 17 80
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20130709/d64a66fb/attachment.html>