[Buildbot-devel] My workaround to multi-repository builds

[Ben Hearsum]

On 09/10/11 07:15 PM, Tom Prince wrote:
> On Fri, 09 Sep 2011 08:49:07 -0400, Ben Hearsum<bhearsum at mozilla.com>  wrote:
>> Hi all,
>>
>> I've been setting up a Buildbot for Flask (http://flask.pocoo.org/) and
>> its supported extensions. For this Buildbot, we're polling both the
>> Flask repository, and each extension's repository. When changes are made
>> to the Flask repository, Builds are run to run Flask's own unit tests,
>> and also to run each of the extension's unit tests against the new Flask
>> code. When changes are made to at extension's repository, its tests are
>> run against multiple versions of Flask.
>>
>> Because we're polling multiple, unrelated repositories (in the VCS
>> sense), and triggering the same builders in multiple different ways, I
>> hit some issues with the Source steps. Specifically: when an extension
>> test builder was triggered from a commit to the Flask repository, the
>> Source step that checked out the extension's code tried to update to a
>> revision only present in the Flask repository.
>>
>> After some head-desking, I found a pretty simple solution: Ignore the
>> SourceStamp completely when its repository doesn't match the one set on
>> the Source step. Concretely, this looks like:
>
> I am guessing you are you Trigger to start builds of Flask from the
> extensions and vice-versa. You can pass alwaysUseLatest=True to start
> the new build without a source stamp, or sourceStamp={ ... dict ...} to
> build a specific version.

Nope, no Trigger or Dependent builders. The extension tests don't depend 
on anything except the Flask source -- so their triggered directly by 
the Scheduler.

> Also, the old source steps accept an alwaysUseLatest option to ignore
> the revision from the SourceStamp. although it appears the new steps
> don't yet support it.

Yeah, I've had to use that, too, for the Flask checkout in extension 
tests. I can't use it for the checkout the extensions, though.

> I know for my own personal projects, there are often multiple
> repositories that I want run builds against. I have one buildbot with
> currently three projects with 3, 5+, and 1 repo respectively.