[Buildbot-devel] multi-repo support: repo URLs

[Dustin J. Mitchell]

On Tue, Apr 13, 2010 at 4:05 AM, Tobias Oberstein
<tobias.oberstein at tavendo.de> wrote:
> I've been thinking about repository identity a little bit.
>
> In particular: whats the identity of a Git repo?

First, this is only one of 12 VC's we have to worry about, so I don't
want to get too Git-specific.  But it's a good question.  Your
approach is fine and fairly rigorous, but probably not germane to most
uses of a DVCS with Buidlbot.  Also, it talked about equivalent
*repositories*, rather than equivalent *URLs*.

I think most groups have one, or a few, "central" repositories from
which Buildbot builds.  This is pretty simple to support, usually with
a post-commit hook or a poller running against that repo, which can
supply an appropriate repository string to allow Buildbot's Source
step to find the code.  So much for the simple case.

In the more complex case, there are lots of repositories, perhaps
running on developers' desktop systems, and you want Buildbot to build
from all of them.  In this case, the repository name from the
perspective of the developer typing "git commit" is not appropriate
for buildbot.  For example, if I commit a change at
  /home/dustin/code/buildbot/t/buildbot
then the Buildmaster at buildbot.net certainly can't check it out at
that URL; instead, it should use something like
 ssh://euclid.r.igoro.us/~dustin/code/buildbot/t/buildbot
This is a fairly straightfoward string transformation that is probably
best performed at the post-commit level.  So here the two URLs are
"equivalent", not because they refer to repos with the same contents,
but because they refer to the *same* repo.

Finally, in the most complex case, if the local repository is not
available, Buildbot needs to be directed to a different repository
where it can expect to find the same source code.  This is obviously
very installation-specific, and in many cases could be worked around
by suitable choice of poller or post-commit hook, but for the moment
let's assume that I have a post-commit hook on my laptop at
  /home/dustin/code/buildbot/t/buildbot
which invokes Buildbot, but since Buildbot does not have any SSH
access to my system, the hook first pushes the branch to
  git at github.com:djmitche/buildbot.git
So buildbot needs to check the source out from
  git://github.com/djmitche/buildbot.git
This is more complicated than the previous case, because we're now
dealing with two different repositories.  However, the complexity is
in making sure that the commit gets pushed to the github repository --
an installation-specific scripting task having nothing to do with
Buildbot -- and the remainder is just another string transformation
that can take place at the hook.

> "Name equivalence" in a strict sense is void, since it puts in "magic"
> knowledge of what repos "usually" contain some change.

So I disagree with this vehemently.  That magic is precisely what we
need, and it is almost always site-specific and thus should be a part
of the configuration.

Dustin

-- 
Open Source Storage Engineer
http://www.zmanda.com