[Buildbot-devel] sharing buildsteps

[Brian Warner]

On Mon, 19 Nov 2007 14:01:19 -0500
"Greg Ward" <gerg.ward+buildbot at gmail.com> wrote:

> I don't know what the Official Recommended Best Practice is, but I
> have been using good old-fashioned procedural abstraction.

FWIW, I've been using a similar approach, where I create the BuildFactories
in a python subroutine in my master.cfg, usually with some parameters that
control the inputs to certain buildsteps. For example, one of them looks like
this:


def make_factory(do_figleaf=False, do_pyflakes_linecounts=False,
                 python=None, trial=None):
    f = factory.BuildFactory()
    f.addStep(Darcs, mode="copy", repourl=SOURCE)
    build_command = ["make", "build", "build-deps"]
    if python:
        build_command.append("PYTHON=%s" % python)
    f.addStep(CompileAndShowVersion, command=build_command)

    if do_pyflakes_linecounts:
        f.addStep(PyFlakes, command=["make", "pyflakes"],
                  warnOnWarnings=True, flunkOnFailure=True)
        f.addStep(LineCount, command=["make", "count-lines"])

    if do_figleaf:
        trial_command = ["make", "test-figleaf"]
        if trial:
            trial_command.append("TRIAL=%s" % trial)
        if python:
            trial_command.append("PYTHON=%s" % python)
        f.addStep(Test, command=trial_command,
                  logfiles={"test.log": "_trial_temp/test.log"},
                  haltOnFailure=True,
                  )
        f.addStep(ShellCommand, command=["make", "figleaf-output"],
                  name="figleaf-html",
                  description=["rendering", "figleaf", "html"],
                  descriptionDone=["render", "figleaf", "html"],
                  )
        f.addStep(PushFigleaf)
    else:
        trial_command = ["make", "test", "REPORTER=--reporter=bwverbose"]
        if trial:
            trial_command.append("TRIAL=%s" % trial)
        if python:
            trial_command.append("PYTHON=%s" % python)
        f.addStep(Test, command=trial_command,
                  logfiles={"test.log": "_trial_temp/test.log"},
                  )
    return f


I then invoke that function a couple of different times. Basically I do some
time-consuming tests on one of our faster platforms, but a baseline set of
tests on all platforms.

I use one builder per platform. My medium-term goal is to establish the
Buildbot convention that Builders are for build-process and platform, not for
branches, and that therefore the recommended practice is to *not* create a
separate Builder for each branch. We need to continue to improve the status
tools to make this actually doable and not too uncomfortable. For example,
you need different Waterfall views (with branch=trunk or branch=YYY) to
distinguish between the most recent build of trunk vs. the branch, and
branch= is CPU-expensive in 0.7.6 . 

So for the moment it's not entirely unreasonable to create a Builder per
branch, and use the Schedulers to make sure that it only runs on changes from
that branch. (note that one problem with this scheme is that when folks hit
the "force build" button, they usually don't bother to fill in a branch name,
which causes your "branch-YYY-only" builder to build trunk, which is usually
kind of confusing).

cheers,
 -Brian