[Buildbot-devel] Slave concurrency?

[Dustin J. Mitchell]

I have what seems to be a more complex buildbots setup than most.  I'm
utilizing the scheduler / builder / buildslave interfaces pretty
heavily to schedule all sorts of different builds at different times.

The upshot is that a single buildslave may have four or five builders
attached to it, and in the current incarnation, that means that the
first buildslave to connect gets assigned all of the builds, while the
other buildslaves sit idle: 

(process/builder.py)
507         # find the first idle slave
508         for sb in self.slaves:
509             if sb.isAvailable():
510                 break

I see "slave-availability + real load-balancing" in the roadmap for
0.7.6 -- is that indicating this situation?

Here's my solution:

Add a state variable to BotPerspective, and pass that object all the way
in to the SlaveBuilder via .attached(...).  Then the SlaveBuilder's
isAvailable can also check its bot's .isAvailable().  

It would be good to have some additional configuration options for
individual slaves, beyond the botname and password.  Might I propose
exposing BotPerspective (under a different name?) into the configuration
file via 'import', and making c['bots'] a list of such objects (with a
compatibility fallback for the existing tuple system).  That is,
master.cfg would contain:

from buildbot.master import Bot

c['bots'] = [
    Bot('host1', 'pass1'),
    Bot('host2', 'pass2'),
    ]

This would allow easy subclassing.  For my purposes, I could use it to
hold extra information about the bots (do they have autotools installed?
are they safe to run a functionality test on?) that can later be used to
assign bots to builders.  I would also override the .isAvailable() to
limit the number of builds that can run simultaneously in VMs on a given
physical host.

Dustin

-- 
        Dustin J. Mitchell
        Storage Software Engineer, Zmanda, Inc.
        http://www.zmanda.com/