[Buildbot-devel] Incremental build with multiple change sources

[Georges Racinet]

Hi,
On Aug 28, 2012, at 9:31 PM, André Anjos wrote:

> We have a project that is scattered through different git
> repositories, say A, B and C.
If you're willing to give the yet-to-be-released 0.8.7 a try, that's  
typically what the new multi-repository features (codebases) can handle.

See below for what can be done in earlier versions (but it's really  
inferior)

>
> The build happens in this order:
>
> 1) check out A
> 2) configure A
> 3) build A
> 4) test A
> 5) check out B
> 6) configure B (at this point, B checks out C automatically - it is a
> buildout recipe using mr.developer)

We also have this latter pattern in different situations (with many  
repositories playing the role of your C). In that case the icing on  
the cake would be to fetch the exact revisions to mr.developer if  
possible,  but I'm not aware of its exact set of capabilities.  That'd  
give you an accurate blamelist.

> 7) build B+C together
> 8) test B+C together
>
> A, B and C use buildbot's github hook to announce changes made on
> their github repositories to buildbot.
>
> A SingleBranchScheduler has "change_filter =
> ChangeFilter(branch='master')" and takes all that in charge.
>
> My problem: When B is updated, the commit hash for the change is
> enforced on the checkout of A and step 1) fails.

On earlier versions of Buildbot, you can use doStepIf to skip the  
wrong updates.
I've written some introductory text about this (see the example with 2  
repos at the end) :
http://www.racinet.fr/index.php?post/2012/04/09/Retour-d-exp%C3%A9rience-sur-Buildbot

BUT it's oversimplified -- I had to add afterwards some really dirty  
magic to make it work in almost all cases (force builds, first builds  
on a given slave).

Going 0.8.7 is certainly the proper way to do that.

Hope this helpgs


>
> Question: What can I do to make this possible? If not, what is the
> closest I can get?



>
> Thanks in advance for any information concerning this,
>
> Best, Andre
>
> -- 
> Dr. André Anjos
> Idiap Research Institute
> Centre du Parc - rue Marconi 19
> CH-1920 Martigny, Suisse
> Phone: +41 27 721 7763
> Fax: +41 27 721 7712
> http://andreanjos.org
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond.  
> Discussions
> will include endpoint security, mobile security and the latest in  
> malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel

Georges Racinet
Anybox SAS, http://anybox.fr
Bureau: 09 53 53 72 97 Portable: 06 51 32 07 27
GPG: 0x33AB0A35, sur serveurs publics