[Buildbot-devel] SlaveLock across several slaves on same computer?

[Dustin J. Mitchell]

On Tue, Jan 5, 2010 at 10:48 AM, Stefan Seefeld <seefeld at sympatico.ca> wrote:
> I don't quite agree. Being able to limit resource usage based on the
> system's load (etc.) is very useful, specifically because the machine
> may be running other things beside a single buildbot slave. (I have
> filed a request for enhancement to support proper load balancing in part
> to address this, many years ago, see issue #42.)

Well, load balancing/limiting is a different issue, and of course
means different things to different people.

We're testing on single-CPU VM's, and our tests are full-bore gcc
compiles, so running two in parallel takes about 2x as long as running
them sequentially.  There's nothing else running on these VMs, so
load-balancing wouldn't help.  When a commit hits, the buildmaster
would see a 0.00 load average, and schedule all of the builds possible
on that build slave immediately.  Six hours later, it might finish
them.

We perform this kind of locking *within* a buildmaster on the master
side.  In fact, we keep track of which hardware each VM is on, and
limit the concurrency on that hardware to CPUs+ε.  But since we have
four buildmasters that don't talk to each other, this still overloads
the hardware when several buildmasters are building simultaneously.

Dustin

-- 
Open Source Storage Engineer
http://www.zmanda.com