[Buildbot-devel] Ideas on builds with multiple variants

[Marc-Antoine Ruel]

Le 7 juin 2010 08:54, Greg Ward <greg at gerg.ca> a écrit :

> Here's the scenario: I've just setup a small BuildBot for testing a
> Mercurial extension that I've written.  Obviously I want to make sure
> that my extension continues to pass its tests with the current stable
> version Mercurial, preferably on multiple platforms.  No surprise
> there -- that's what BuildBot is for.
>
> But it's more interesting than that: I also want to ensure that my
> extension works with several versions of Mercurial.  The docs claim
> that it supports Mercurial 1.3 and up, but any commit to my extension
> could break that promise.  So what I want is something like
>
>  for hgversion in (1.3 1.4 1.5); do
>    test my extension with hg${hgversion}, assuming it's available on $PATH
>
> (The idea is that build slave maintainers could install
> /usr/local/bin/hg{1.3,1.4,1.5} as aliases for 1.3.x, 1.4.y, and 1.5.z,
> the latest versions on each respective stable branch.)
>
> But wait, it gets more fun: any commit to Mercurial itself could break
> my extension.  This is most likely to happen on Mercurial's trunk (aka
> "default" branch), which is going to become Mercurial 1.6.  So really
> I should add
>
>  cd <mercurial source tree>
>  hg pull
>  hg update
>  make local
>  test my extension with the hg just built
>
> after the above loop.  Or as part of it.
>
> I can think of lots of ways to implement this, but I can't see that
> one way is obviously superior.  Possibilities:
>
>  1) put the loop over $hgversion in master.cfg and require build
> slave maintainers to set things up "just so" (probably OK because I
> plan to be the only slave maintainer, but it still feels brittle)
>  2) keep the loop over $hgversion in master.cfg, but put a script in
> my source tree that takes $hgversion as an argument and ignores
> missing Mercurial versions (same as #1 but a little less brittle)
>  3) move the loop over Mercurial versions into my extension's source
> tree, and just have BuildBot call that script
>  4) keep a clone of Mercurial's source tree on the slave and use it
> to build every required hg version for every build (requires two
> source update steps, which is a bit weird but seems to work fine...
> but it's a lot of wasted work, since Mercurial 1.3.1 is not going to
> change)
>
> Now before you think "bah, this is easy", here's my next thought: my
> extension also claims to support Python 2.4 .. 2.x.  But any commit
> could break that promise.  So really, I should *also* test with a
> range of Python versions:
>
>  for pyversion in 2.4.x 2.5.x 2.6.x trunk:
>    for hgversion in 1.3.x 1.4.x 1.5.x trunk:
>      build Mercurial $hgversion with Python $pyversion
>      run my extension's tests with $hgversion and $pyversion
>
> Now things get *really* fun: do I build a new Python on every slave
> for every build?  Require multiple Python versions on $PATH?
>
> Has anyone setup something similiar?  Ideas, thoughts, advice?  Thanks --
>
> Greg


I'd personally have a Builder with the (2.4, 2.5, 2.6) * (1.3, 1.4, 1.5)
matrix and another try with ('trunk') * ('trunk') just to be sure stuff
would mess too much, but that's me. Also since building python would slow
things down, I assume building hg is a no-op, it's better to parallelize a
bit.

So I'd create two new BuildFactory; one to generate 9 test BuildStep for
BuilderA and Compile+Compile+test for BuilderB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20100607/5a89d720/attachment.html>