[Buildbot-devel] Add "latent slave" capability to buildbot (e.g., on-demand ec2 slaves)

[Jean-Paul Calderone]

On Tue, 06 Jan 2009 10:19:41 -0500, Gary Poster <gary.poster at canonical.com> wrote:
>We (Canonical/Launchpad) are interested in adding what I'm calling
>"latent slaves" to buildbot--a class of slaves that can start up
>external machines like Amazon EC2 boxes when a test needs to be run.
>
> [snip]
>
>Discussion:
>
>- (re: 1. [MINIMAL]) The slickest way to set up a real slave would be to
>have the LatentBuildSlave install buildbot on the image itself, but that
>sets up a lot of expectations that would be difficult to fulfill across
>operating systems.  A simpler improvement to the minimal approach is to
>have a convention for images to follow--some mechanism to send botname,
>password, master ip, and port to an instance, and to ask the image's
>buildbot to start up.  A Twisted pb server waiting for data?  Or perhaps
>you provide a configuration script to the buildslave?  I think I'll
>start with the minimal approach and see what develops.

I don't know if this helps at all, but keep in mind that PB is basically
symmetric.  If you can connect to a process running on the newly booted
instance, then you're essentially done as far as connection setup needs
go.  It would take a little bit of tweaking to get a regular client/server
conversation going over such a reverse connection, but it really should be
just a little.  Then you can skip the step where you tell the slave the
botname, password, master ip, and port number (although perhaps you want
the slave to have a shared secret so it will only talk to your master).

Jean-Paul