[Buildbot-devel] monitoring and tagging multiple git repositories

[Dickon Reed]

Hello,

I am involved with a project that uses buildbot and OpenEmbedded to build a
product from about 90 git repositories.  For the last few months we have
used a PBChangeSource that is contacted
by src/master/contrib/git_buildbot.py from the git post-receive hooks
(modified to include the repository path). Then, we have one builder per
branch which builds from the HEAD of that branch across all the
repositories. A build steps tags the revisions it is building across all
repositories. A problem with the arrangement is that typically the builds
don't get around to checking out HEAD and tagging what they are building for
a few minutes. In that time it is common for other changes to be pushed and
so included in the build. Therefore in many cases what actually gets built
and tagged does not match the buildbot change records. Another problem is
that some people push over NFS from machines that do not have enough of a
python environment installed to run the push hook, so their pushes do not
trigger builds.

One option I am considering is to implement a change source that polls for
untagged revisions across all branches in all repositories, and tags
branches that have new revisions in all repositories. It would then invoke
addChange with the revision being the name of the tag, and the comment
containing the git log output for the new revisions introduced by that tag.
There are some sketches along these lines at
https://github.com/dickon/buildbot in src/master/buildbot/changes/multigit.py,
which has unit test coverage. However, this is all pretty early on and I
would be very interested to hear about any experiences people have had with
this kind of approach.

Another option is repo/gerrit. However, that feels like a bigger change to
the work flow of the developers than we need to impose to resolve the
problems I mentioned above.

Thanks,

Dickon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110607/8983fc3f/attachment.html>