[Buildbot-devel] RFE: load management for build slaves

[Stefan Seefeld]

hi there,

I'v been using buildbot 0.7.2 quite successfully for a couple of weeks now.
It's a great tool to automate build infrastructure transparently.

As we are looking into moving more of our build infrastructure to buildbot,
I'm a little concerned about scalability. Other then explicitely setting
build times (via Periodic or Nightly) there doesn't appear to be any way
to manage buildslave loads.

I'm thus looking for ways to improve this situation, and I can see two
alternative approaches:
1) limitting concurrency on a slave and
2) schedule based on current load

Both have advantages and disadvantages.

Limitting concurrency:

A slave would have some 'concurrency' number assigned to it, reflecting
how many parallel builders are allowed to run on it in parallel. In its
simplest implementation, a slave would queue builders sent to it to
enforce that limit. Alternatively, the buildmaster / scheduler could
do the queuing on the master side.

While this seems reasonably straight forward to implement (at least the
slave-side queuing), the obvious drawback here is that the logic breaks
down as soon as there are multiple slaves on the same host.

Load balancing:

A slave provides the means for the master to query a 'load level', based
on which the master decides whether or not to run a builder there.
While it would be possible to just queue tasks and then ping slaves
regularly to watch for better times, I believe the best way to support
load balancing would be to allow multiple slaves to be classified equally,
so that builders can run on any slave in a given class. So instead of
assigning builders to slaves, slaves would get some new 'host type' (say)
attribute, to which builders then get assigned. Then the owning scheduler
can pick the best host to run the builder on, or queue it if none is
available.

What do people think about the above ? Would this be a valuable addition
to buildbot ? What is the best way to implement it ? How could it be
broken down into subtasks ?

Regards,
		Stefan