[Buildbot-devel] multi-repo support: repo URLs

[Tobias Oberstein]

To support scenarios where multiple source repositories are involved, a couple
of questions/issues with BuildBot exist.

I'll raise them as I see them in different posts.

This one is about the "repo URL" question.

=================================================================================

Obviously, when multiple repos are involved, it is no longer sufficient/convenient
to define the repository URL (like for Git ssh://domain.tld/somepath) on the
VCS sourcing build step as is done with BuildBot 0.7.

My view is: the repo URL is logically part of the sourcestamp. So I extended the
git_buildbot hook to be able to take a

--repository=...

parameter, and modified the source build step (while changing the signatures of
funs).

Then, Benoît stepped in, and his view is slightly different. He paraphrased changes
made by Marcus, which allow the repoURL to be still defined on SourceStep, but
contain a "%s" which gets filled if the source sends a repository in the change.
Also he modified the VCS source steps taking into account the above without having
to change any fun signatures.

Also he made comments which led me to improve the git_buildbot hook: it determines
the Git repo filesystem path on it's own (no need to give on cmd line), and now
only optionally takes a git_host parameter.

This latter version of git_buildbot hook isn't merged into mainline. You can get it
here:
http://github.com/oberstet/buildbot/commit/7ae6b064869fcb0a4134353c3767d39099f1918e

* flexible options for controlling repository sent
    --repository    : provide fully qualified Git repo URL (will override 
    the following)
    --git-host      : provide Git repo host, will get concatenated with 
    CWD to
                      sent fully qualified repo URL
    no options used : will just send CWD (local Git repo filesystem path) 
    as repository,
                      still allowing to construct full qualified repo URL 
                      on BuildBot side

=================================================================================

Thus, in effect, you have now 3 options:

A) fixed, fully qualified repo URL on source step

git_buildbot
+
build.addStep (Git (repourl = "ssh://bldslave@domain.tld/home/someone/somerepo"))


B) only base repo URL on source step, filesystem path of repo sent in change

git_buildbot
+
build.addStep (Git (repourl = "ssh://bldslave@domain.tld%s"))


C) repo base URL and repo FS path sent in change

git_buildbot --git-host= ssh://bldslave@domain.tld
+
build.addStep (Git ())

=================================================================================

I am fine with the options, since both B) and C) allow me to do what I want.

I go with C) for reasons I explain.

As said, I think the fully qualified repo URL should be part of a change, 
since together with branch/revision, it uniquely identifies the change.

When using B), the fully qualified repo URL is only derived in an actual build,
since it needs the buildstep context to concatenate the base URL and the
repo path from the sourcestamp.

This means e.g. that the sourcestamp stored in the database does not contain the
fully qualified repo URL.

Also, at least the web status page for the change does only give the sourcestamp
repo path. Maybe this can be fixed.

If the hook doesn’t send the fully qualified URL, then I think the right place
to derive the fully qualified URL is not the build step, but the change source.

e.g.

c['change_source'] = PBChangeSource(repourl = "ssh://bldslave@domain.tld%s")

This would allow for already storing the fully qualified URL in the sourcestamp
in the database.

But: what do you do if you have more than one Git host?

You needed more than one change source ... one per Git host.

So:

c['change_source'] = []
c['change_source'].append (PBChangeSource(repourl = "ssh://bldslave@githost1.tld%s", port = 9000))
c['change_source'].append (PBChangeSource(repourl = "ssh://bldslave@githost2.tld%s", port = 9001)) 

From a design perspective, IMHO this would be clean.

On the other hand: in practice, I don't need it, since I can do C).