[Buildbot-devel] monitoring and tagging multiple git repositories

[P. Christeas]

On Tuesday 07 June 2011, Dickon Reed wrote:
> I am involved with a project that uses buildbot and OpenEmbedded to build a
> product from about 90 git repositories.  
> ...
> A problem with the arrangement is that typically the builds
> don't get around to checking out HEAD and tagging what they are building
> for a few minutes. In that time it is common for other changes to be
> pushed and so included in the build. Therefore in many cases what actually
> gets built and tagged does not match the buildbot change records. ...

This scenario reminds me of the proposed structure I had asked OpenWRT 
developers to follow, some years ago:
  _use git submodules_

Simply, much before buildbot has appeared in our horizon, my concern had been 
how to sync several projects into the group (called "distro" or "main 
project"). Sumbodules of git offers exactly that.

Now, this could be a candidate solution to your situation, too (with all due 
respect to other proposals and technical issues).

Personally, I use that in my openerp repo (but not the one that feeds 
buildbot, so far) :
http://git.hellug.gr/?p=xrg/openerp;a=tree

See, the idea is to have /controlled versioning/ of each sub-component of your 
platform project, that is then reproducible. Example, I want to commit 
libhal0.5.12 as my version of choice, and then build everything against that. 
One day later, I push libhal0.5.14 and retry the whole platform, for 
regressions.

git-submodules guarrantee exactly that, that each one of the sub-projects will 
be at a specific commit at the time of build (or even, at the time being hacked 
by your project developer)

Regarding hg repos, I would recommend using fast-export/fast-import in a set 
of automated scripts (done that in a Makefile for openerp), so that you bring 
all your code into git repos for symmetry.

Regarding the main git repo, you'd need a few simple bash scripts to 
orchestrate automatic sync of sub-projects (example, in the case you want all 
subprojects to /blindly/ fetch their latest version), and generating the 
commit message. Otherwise, the manual method works out of the box.

Regarding buildbot, we might need just some extra steps and minor additions to 
support submodules (do we already?) . Git will do the rest.


-- 
Say NO to spam and viruses. Stop using Microsoft Windows!