[Buildbot-devel] Determining steps based on what's checked out...

[Dmitry Nezhevenko]

On Fri, Mar 16, 2012 at 01:14:42PM -0700, Jarod McBride wrote:
> So let's say I have a project that has a list of components in it.
> There's a folder for each component and a set of unit tests.  Each
> component's test can be run by calling...
> 
> phantomjs foo/build/scripts/jasmineRunner.js http://localhost:4020/"component name"/current/en/tests/unit.html
> 
> So I'd like to have a step that looks at how many components there are
> in the git checkout, then creates a buildstep for each component and
> runs that unit test based on the name of the component folder.  Like I
> said, I can pawn the work off to a shell script to do it but then I
> don't get the same level of reporting or control in buildbot over those
> individual steps.
> 
> Is this possible now or can BuildBot gain a feature to make this sort of
> thing happen?

It's definitely not possible "out of the box" (by just writing master.cfg
file) but I see a few ways to handle this:

1. It's possible to implement own build factory (as a replacement to
default BuildFactory that is described here:

http://buildbot.net/buildbot/docs/latest/manual/cfg-buildfactories.html

You'll need to reimplement newBuild() method. Most problematic part is
that factory need to know actual build steps before build starts. Also
it's executed on master so you need to access your VCS somehow. If your
project file is available via HTTP (maybe using VCS web interface, or it's
possible to use hook to copy such file somewhere on every change) you
probably can just grab it from there, parse and create required steps. But
downloading it will take some time and will block whole master (including
WebStatus and other builds). So you probably will need to patch buildbot
code so that it'll be allowed to return Deferred from newBuild method.

This is a bit tricky but probably most clean way to implement such
feature.

2. If you really can't determine your build steps without checkout of
whole source tree, you can try to create two builders. First one will
have just one step that will parse required files and trigger another
Builder with all required information in BuildRequest. But anyway second
Builder will require own Factory that can parse such build requests and
create proper build steps.

First idea looks much cleaner for me.

-- 
WBR, Dmitry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://buildbot.net/pipermail/devel/attachments/20120317/9033a0d9/attachment.bin>