[Buildbot-devel] Add "latent slave" capability to buildbot (e.g., on-demand ec2 slaves)

[Gary Poster]

On Tue, 2009-01-06 at 21:50 -0500, Jean-Paul Calderone wrote:
> On Tue, 06 Jan 2009 10:19:41 -0500, Gary Poster <gary.poster at canonical.com> wrote:
> >We (Canonical/Launchpad) are interested in adding what I'm calling
> >"latent slaves" to buildbot--a class of slaves that can start up
> >external machines like Amazon EC2 boxes when a test needs to be run.
> >
> > [snip]
> >
> >Discussion:
> >
> >- (re: 1. [MINIMAL]) The slickest way to set up a real slave would be to
> >have the LatentBuildSlave install buildbot on the image itself, but that
> >sets up a lot of expectations that would be difficult to fulfill across
> >operating systems.  A simpler improvement to the minimal approach is to
> >have a convention for images to follow--some mechanism to send botname,
> >password, master ip, and port to an instance, and to ask the image's
> >buildbot to start up.  A Twisted pb server waiting for data?  Or perhaps
> >you provide a configuration script to the buildslave?  I think I'll
> >start with the minimal approach and see what develops.
> 
> I don't know if this helps at all, but keep in mind that PB is basically
> symmetric.  If you can connect to a process running on the newly booted
> instance, then you're essentially done as far as connection setup needs
> go.  It would take a little bit of tweaking to get a regular client/server
> conversation going over such a reverse connection, but it really should be
> just a little.  Then you can skip the step where you tell the slave the
> botname, password, master ip, and port number (although perhaps you want
> the slave to have a shared secret so it will only talk to your master).

Hm, thank you.  That would be really cool, and a nice generic approach.
I don't know that code but after I get the minimal story working, that
sounds like the most interesting approach to pursue next.

Gary