[Buildbot-devel] [Python-Dev] buildbot

[Brian Warner]

> Currently, branches are not supported, because buildbot is
> somewhat limited. When I get a solution for this problem,
> I plan to have all buildbots build both the trunk and the
> 2.4 branch; such builds would only be initiated whenever
> something is committed on the branch.

Branch support appeared fairly recently (buildbot-0.7.0, about two months
ago), and I believe it should be sufficient to trigger builds for both the
trunk and the 2.4 branch. The last part of the setup involves modifying the
svn_buildbot.py hook script, though, so I won't claim this support is
complete yet.

The basic idea is that you create multiple Scheduler instances, one for each
branch you want to monitor. Both Schedulers are configured to trigger all the
Builders. You configure the SVN-checkout step of the build process with a
baseURL= argument to provide the common portion of the SVN URL, and add a
defaultBranch= argument to point to the trunk. Each Build is done for a
specific branch, and the SVN checkout step just appends the branch name to
the baseURL to figure out what to pass to 'svn co %s'

Assuming the ChangeSource is providing branch information properly, this will
result in any change to the trunk causing a trunk build, and any change to
the 2.4 branch causing a 2.4 build.

Your master.cfg file will probably want to have some of the following
elements:

---BEGIN---
from buildbot.changes.pb import PBChangeSource
c['sources'] = PBChangeSource() # N.B.: don't set prefix=

from buildbot.scheduler import Scheduler
all_builders=["sparc solaris10 gcc trunk",
              "g5 osx trunk",
              "amd64 gentoo trunk",
              "x86 gentoo trunk"]
s1 = Scheduler("trunk scheduler", "trunk", 2*60, all_builders)
s2 = Scheduler("2.4 scheduler", "branches/release24-maint", 2*60, all_builders)
c['schedulers'] = [s1, s2]

...
from buildbot.process.factory import s
from buildbot.process.step import SVN
checkoutstep = s(SVN, baseURL="http://svn.python.org/projects/python/",
                 defaultBranch="trunk", mode="copy")
steps = [checkoutstep, ...]

---END---


The tricky part is getting branch-annotated Changes into the buildbot in the
first place. I assume you're using the svn_buildbot.py script. The version
provided in the current release always sets the branch to 'None', which
implies the default branch. You will have to edit this script to give it the
ability to decide which branch each change is for (by adding some code which
does a test like filename.startswith("trunk/"), perhaps), then include the
branch name in the RPC call that informs the buildmaster about the new
Change. Basically you'll need to change this call:

        return persp.callRemote('addChange', {'who': who,
                                              'files': changed,
                                              'comments': message,
                                              'revision': revision})

into one like:

        branchname = "trunk" # or "branches/release24-maint"
        return persp.callRemote('addChange', {'who': who,
                                              'branch': branchname,
                                              'files': changed,
                                              'comments': message,
                                              'revision': revision})

There's been some discussion on the buildbot-devel list about the best way to
implement branch-determination. One school of thought says it belongs close
to the repository (hence inside a hook script like svn_buildbot.py). Another
says it is easier to configure from the buildbot side, in which case you'd
probably want to modify buildbot.changes.pb.PBChangeSource to analyze the
filenames and decide which branch they live on there.

There was also a patch[1] to add some regexps to svn_buildbot.py to do this
kind of branch determination. I haven't been able to review it properly yet,
but it may give you some ideas. svn_buildbot.py certainly needs help here.
The rest of the Buildbot is branch-enabled and ready to go. The only
lingering issues are with status delivery: the main HTML "Waterfall" HTML
will interleave builds of both branches on the same page, which could be a
bit confusing (if the top line is red, does that mean the trunk is broken, or
the 2.4 branch?). Fixing this (by creating separate pages for each branch) is
high on the TODO list.


If there's any way I can help out further, please let me know. I'm ecstatic
that Python is using a buildbot.. if you have any feature suggestions or bug
reports, please send them to me, to the buildbot-devel mailing list[2], or
file them in the sf.net bugtracker[3]. Or corner me at PyCon, or in #twisted
(where I can sometimes be found as <warner>).

thanks!
 -Brian  (author of Buildbot)


[1]: http://sourceforge.net/mailarchive/message.php?msg_id=13977004
[2]: http://sourceforge.net/mailarchive/forum.php?forum=buildbot-devel
[3]: http://sourceforge.net/projects/buildbot/