[Buildbot-devel] Starting slaves on demand?

[Jean-Paul Calderone]

On Sun, 7 Dec 2008 13:29:05 +0100, Axel Hecht <l10n.moz at googlemail.com> wrote:
>I don't think that this is within the design of buildbot. The infrastructure
>that buildbot uses to dispatch work onto a slave is that the buildbot slave
>is running there.
>
>Anything else seems to be out of scope.
>
>For the pipol use-case, I'd suggest to have a builder that runs on a
>priviledged slave (maybe the same machine as the master) that goes through
>the steps of reserving and booting, and then triggers a dependent scheduler
>once the relevant slave is up.

Another possibility would be to have a slave implementation which only acts
as a proxy.  When it is asked to start a build, it does something to start
up the real slave, then forwards all commands from the server and responses
from the slave to the other end.  When the build finishes, it can shut down
the real slave.  Such a slave could run on the same host as the master since
it wouldn't add significant additional load, and multiple real slaves could
even be proxied through a single instance of such a slave (at worst, you'd
need multiple TCP connections, one for each slave, but you might even be
able to do it all over a single TCP connection).  And... it could even run
in the same process as the master, if you wanted.  Keep pushing the logic
closer to the master's notion of slave state tracking and eventually it
might even turn into a coherent extension of buildbot's notion of s slave
(ie, perhaps eventually it could be a different implementation of BuildSlave
in the master process which knows there isn't a persistent TCP connection to
the real slave and deals with this issue in a way that still allows builds to
be scheduled on that slave).

Jean-Paul