[Buildbot-devel] buildbot and a cluster (dynamically controlling buildslaves)

[Derek Graham (ML)]

Hello all,

I want to use buildbot with a compute cluster that is only accessible
through a job scheduler software framework (Platform LSF in case you've
heard of it), which means that the machine your job gets to run on is picked
by this software.

What part of buildbot do I need to customise to allow me to dynamically
control which buildslaves to pick? I believe I can't create a new BuildStep
or latent BuildSlave since that runs on a buildslave so one has to be
already chosen.

This is what I was thinking: I pass a function to nextSlave of Builder that
returns the chosen machine. Since picking the machine may take some time, I
fork off a new thread and return None. In this new thread I call the command
that submits a job (with resources that have been specified to Builder using
Properties) that runs sleep (this reserves this machine for me). I then poll
LSF to find out what machine it has been allocated and then call
addLatentSlave() on the Builder. This then eventually calls Builder.run()
(does it?) and runs the SlaveBuilder.

What do you think?

If this could work, then 1) can I execute a external command from the
master.cfg? and 2) How do I create a LatentSlaveBuilder from the machine
name?

Thanks for you help,

d
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20100910/575b4d5c/attachment.html>