[Buildbot-devel] How to execute two shell commands from a source buildstep

[Brian Warner]

On Thu, 25 Oct 2007 12:12:38 -0400
Casey Dahlin <cdahlin at redhat.com> wrote:

> I am trying to rewrite the Git interface to buildbot so that it doesn't 
> rely on cogito and is a bit more flexible. I need to tell the buildslave 
> to execute two shell commands in sequence when it checks out the project 
> for the first time. How do I do this?  

To make a single BuildStep execute multiple shell commands, there are two
approaches. The first is to do all the work on the buildmaster side, the
second is to do the work on the buildslave side.

The first approach has the advantage of not requiring buildslave changes,
which means you can use old buildslaves. Each BuildStep can trigger zero or
more RemoteCommands, and RemoteShellCommand is a kind of RemoteCommand.
Unfortunately, I don't know if we have any examples of this sort of thing in
there right now. I am planning to clean up the way RemoteCommands are started
(the goal is to make it behave just like regular twisted Deferred chains),
but the changes got too destabilizing to make it into 0.7.6 .

The second approach (which is currently used by all the source-checkout
commands) takes advantage of the fact that each RemoteCommand (on the
buildslave side) can trigger zero or more actual shell commands. This
approach gives you better access (you can os.stat() files directly, for
example) and offers marginally faster operations (fewer roundtrips), at the
expense of only working with a buildslave that's new enough to have your new
code. All of the Source steps use this, so look at those for examples, like
buildbot.slave.commands.SourceBase and .SVN .

The SourceBase class has methods for doing a full checkout and for doing an
update: the VC-specific subclasses fill in those methods. SourceBase tries to
do an update, and if that fails it triggers the full-checkout method.

I'd recommand doing it on the slave side, and have the Git full-checkout
method do a chain of shell commands. I think the old Arch/Baz classes need to
do something like this, so check them out for some hints.

cheers,
 -Brian