[Buildbot-devel] database-backed status/scheduler-state project

[Axel Hecht]

2009/9/3 Ben Hearsum <bhearsum at mozilla.com>

>
> On 3-Sep-09, at 6:07 AM, Axel Hecht wrote:
>
>>
>> A big load issue right now are the changesources, but I got those covered.
>> If there are more master load issues, why not spin more? Riiiight, let's
>> detail on that.
>>
>> We're not having any load issues on the slaves. We may have not enough
>> slaves, and disks, but in general, each slave is fine. That's because
>> we're
>> using generic slaves per platform that can do basically any build on that
>> platform, independent of branch. We're setting the builds limit to 1 per
>> slave, and our slaves build what needs to be built.
>>
>> Adding masters breaks that.
>>
>
> I don't think it does. In the plan Brian describes every master would be
> identically configured - able to handle any job from any branch. It allows
> us to end up with a bunch of masters in different places with smaller slave
> pools. (Does this belong on a Mozilla list rather than Buildbot?)
>

My point was that adding masters as they ship today breaks things.

I do think that the proposed scheme can solve that, I just think that you
get the same net result by making slave pools work across masters, which
sounds like an easier patch to me.

To rephrase how I understand the two ways:

In Brian's proposal, you'd have a slave-master per N slaves, which picks up
build requests from a single scheduling master.
In my proposal, you have, say a master per branch in our case, and they'd
share slave pools.

I fear the edges in fighting race conditions in the scheme where you have
decoupled daemons trying to fight over rows in a database, tbh.

Fighting for a maxbuilds lock on the slave on the other hand seems rather
easy inside the single threaded twistd slave daemon.

The status reporting infrastructure is mostly untouched by which you try, as
are the benefits of a log server.

Reading bug reports, I see more common use in shared slaves than in
hierarchical and split masters, too.

Axel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20090903/91fa6e4b/attachment.html>