[Buildbot-devel] [Use Case] Best way to build the same code with several configurations?

[John Hopkins]

On 13-05-30 10:05 PM, Charles Lepple wrote:
> On May 30, 2013, at 2:24 PM, sergio borghese wrote:
>
>> Creating a distinct builder per configuration, sound a bit overkill, as the steps run by each builder would be the same, but the initial configuration one.
> Since master.cfg is a Python script, you can reduce duplication by wrapping the step creation process in a 'for' loop (iterating over the configurations, and leaving the other steps constant).
>
> That said, others have recommended keeping as much of the build information in the source tree itself (one or more scripts which will kick off each library configuration).
>
> The "best" way to handle this is subjective, and it depends on whether you prefer having all of the script output glommed into one or more generic steps, or if you want to be able to parse out individual build outputs and success/failure indicators. I have worked on projects where nobody else wanted to touch the Buildbot configuration, so we gravitated towards scripts checked into the source tree. I have also worked on systems that kept the Buildbot configuration itself in a distributed version control system, and each step was programmatically generated. Each team member had to learn the Buildbot configuration language, but it also provided finer-grained progress information for long-running builds. It probably helped that we had many master/slave pairs operating in parallel, so restarting one master did not affect other builds.
A couple of other use-cases for more granular builds:
- Using separate buildbot builders can enable parallel builds (eg. debug
and dist builds can run in parallel).
- If a packaging process fails due to a transient error (say, DNS goes
down) and runs in its own buildbot builder, that packaging build can be
resubmitted without having to run the compilation process again.

I agree that a good design philosophy (regardless of whether you choose
a monolithic build or more granular builds) is to put most of the build
logic outside of your buildbot factories.

John