[Buildbot-devel] buildbot-0.6.0 has been released

[Brian Warner]

I've just finished getting 0.6.0 out the door. Many thanks to everyone who
contributed to this release! The (signed) tarball is available from the
sourceforge download page, at:

 http://sourceforge.net/project/showfiles.php?group_id=73177

It is signed with my GPG key as always, available from
http://www.lothar.com/warner-gpg.html . (This is the third release signed
with my new GPG key 0x1514A7BD). The md5sum of the tarball is:

 3d526c7898edfda40a28708c68d33e36  buildbot-0.6.0.tar.gz


This release updates nearly every subsystem of the buildbot, from
configuration files to status delivery to results storage. The highlights:

 config file: all "status targets" (the web page, the IRC bot, and the new
              MailNotifier email-sender) are now added to a list in
              master.cfg named c['status']. the c['webPort'], c['webPath'],
              and c['irc'] options are deprecated in favor of doing
              c['status'].append(html.Waterfall(options)), etc.

              Build configuration has been cleaned up.
              ConfigurableBuildFactory is gone, and has been replaced with
              buildbot.process.factory.BuildFactory as a base class.
              factory.py has classes for a number of common build schemes
              (autoconf, CPAN, distutils), each of which accepts a 'source='
              argument that defines a BuildStep which retrieves the sources.

              All SourceSteps now use the same set of basic modes: "update",
              "copy", "clobber", and "export". Check the docstring for
              buildbot.process.step.Source for details.

              Internally, it is now much easier for a BuildStep to fire off
              multiple RemoteCommands, and (on the slave side) for a single
              RemoteCommand to fire off multiple ShellCommands. This makes it
              easier to implement steps like Trial, which needs to do
              multiple commands and retrieve multiple results files.

              Sending a SIGHUP to the buildmaster now causes it to re-read
              its config file.

 status delivery: the new MailNotifier class can be used to send email after
                  each build. Messages can be sent after all builds, after
                  failing builds, or after "newly-failing" builds (those
                  which fail when the previous one had succeeded). The mail
                  can be sent to a fixed list of recipients, to the people
                  that were involved in the build (i.e. the submitters of any
                  Changes that were included in this build), or both. The
                  mail can include full build logs or be a simple brief
                  "passed/failed" note.

                  The "force build" button on the HTML status page can be
                  disabled by passing allowForce=False to the
                  html.Waterfall() constructor. The same argument can be
                  passed to words.IRC to disable this command in the irc bot.
                  Thanks to Fred Drake for code and development work.

 results storage: the results of each build (both pass/fail and the contents
                  of the build logs) are stored in a separate pickle-format
                  file on the buildmater. (0.5.0 stored them in a bunch of
                  subdirectories, as in buildNN/stepNN/logNN, whereas 0.6.0
                  just uses buildNN). The accumulated Changes are kept in a
                  changes.pck file. The net result is that the
                  buildbot-shutdown.tap file (produced by twistd upon
                  shutdown, and containing the serialized buildmaster
                  instance) is no longer necessary to restart the
                  buildmaster. You can 'twistd -f buildbot.tap' with the same
                  .tap file you started with the last time, and as long as the
                  build objects are still on the disk where they were left,
                  no status history will be lost.

 New VC steps: we now have basic support for Darcs and Arch source trees,
               and (thanks to Dave Peticolas) even limited support for
               Perforce. Source-tree checkouts are performed with a -D
               TIMESTAMP or -r REVISION parameter to fetch a tree that
               exactly matches the latest Change that is included in the
               build. This removes a race condition that can result in
               mismatched sources being used for a build, especially when
               using high-latency notification schemes like email-based
               ChangeSources.

 Buildslaves: the VC checkout commands on the slave side are smarter. All
              slave commands are versioned, and the version numbers are sent
              to the master upon request, which will make it easier to
              provide backwards-compatibility with old buildslaves in the
              future.

 Invocation: there is a new /usr/bin/buildbot command, which can be used to
             create buildmasters and buildslaves (each in their own
             directory), and will start and stop them (basically a front-end
             to mktap and twistd). This will expand in the future, to provide
             an interface to the 'try' feature, and to run a basic status
             client which can contact the build master and query for build
             results.

 Debian Packaging: thanks to Kirill Lapshin, we have an initial set of
                   debian/rules scripts to produce .deb packages. They need a
                   lot of testing, so pre-built .debs have not been produced
                   for 0.6.0, but you can make your own with 'debuild'. The
                   idea is to have /etc/default/buildbot provide a list of
                   directories where masters and slaves are configured, then
                   switch to a new 'buildbot' user and launch each one at
                   boot time (through an init.d script). I'm not sure the
                   "switch to the 'buildbot' user" part is working yet.


The bad news: you will probably need to edit your config files to handle the
new BuildFactory classes, and you will need to make new buildbot.tap files
(which means losing your build history). The new history pickles mean that
there shouldn't be any history-losing changes in the future, and the new
configuration format is easier to deal with than before, so I think the
upgrade effort will be time well spent. But it may be a good idea to set up a
new directory to test out 0.6.0 before moving your production systems over to
it.

When starting the buildmaster, I recommend watching twistd.log closely for
complaints about incorrect syntax in the master.cfg file. Once builds have
started, twistd.log becomes a bit too noisy to see deprecation warnings and
such.


The full NEWS file is attached below. Please send all questions, success
stories, bugs, etc, to the buildbot-devel list as usual.

Have a hypothetical day,
 -Brian



* Release 0.6.0 (30 Sep 2004)

** new features

*** /usr/bin/buildbot control tool

There is now an executable named 'buildbot'. For now, this just provides a
convenient front-end to mktap/twistd/kill, but eventually it will provide
access to other client functionality (like the 'try' builds, and a status
client). Assuming you put your buildbots in /var/lib/buildbot/master/FOO,
you can do 'buildbot create-master /var/lib/buildbot/master/FOO' and it will
create the .tap file and set up a sample master.cfg for you. Later,
'buildbot start /var/lib/buildbot/master/FOO' will start the daemon.


*** build status now saved in external files, -shutdown.tap unnecessary

The status rewrite included a change to save all build status in a set of
external files. These files, one per build, are put in a subdirectory of the
master's basedir (named according to the 'builddir' parameter of the Builder
configuration dictionary). This helps keep the buildmaster's memory
consumption small: the (potentially large) build logs are kept on disk
instead of in RAM. There is a small cache (2 builds per builder) kept in
memory, but everything else lives on disk.

The big change is that the buildmaster now keeps *all* status in these
files. It is no longer necessary to preserve the buildbot-shutdown.tap file
to run a persistent buildmaster. The buildmaster may be launched with
'twistd -f buildbot.tap' each time, in fact the '-n' option can be added to
prevent twistd from automatically creating the -shutdown.tap file.

There is still one lingering bug with this change: the Expectations object
for each builder (which records how long the various steps took, to provide
an ETA value for the next time) is not yet saved. The result is that the
first build after a restart will not provide an ETA value.

0.6.0 keeps status in a single file per build, as opposed to 0.5.0 which
kept status in many subdirectories (one layer for builds, another for steps,
and a third for logs). 0.6.0 will detect and delete these subdirectories as
it overwrites them.

The saved builds are optional. To prevent disk usage from growing without
bounds, you may want to set up a cron job to run 'find' and delete any which
are too old. The status displays will happily survive without those saved
build objects.

The set of recorded Changes is kept in a similar file named 'changes.pck'.


*** source checkout now uses timestamp/revision

Source checkouts are now performed with an appropriate -D TIMESTAMP (for
CVS) or -r REVISION (for SVN) marker to obtain the exact sources that were
specified by the most recent Change going into the current Build. This
avoids a race condition in which a change might be committed after the build
has started but before the source checkout has completed, resulting in a
mismatched set of source files. Such changes are now ignored.

This works by keeping track of repository-wide revision/transaction numbers
(for version control systems that offer them, like SVN). The checkout or
update is performed with the highest such revision number. For CVS (which
does not have them), the timestamp of each commit message is used, and a -D
argument is created to place the checkout squarely in the middle of the "tree
stable timer"'s window.

This also provides the infrastructure for the upcoming 'try' feature. All
source-checkout commands can now obtain a base revision marker and a patch
from the Build, allowing certain builds to be performed on something other
than the most recent sources.

See source.xhtml and steps.xhtml for details.


*** Darcs and Arch support added

There are now build steps which retrieve a source tree from Darcs and Arch
repositories. See steps.xhtml for details.

Preliminary P4 support has been added, thanks to code from Dave Peticolas.
You must manually set up each build slave with an appropriate P4CLIENT: all
buildbot does is run 'p4 sync' at the appropriate times.


*** Status reporting rewritten

Status reporting was completely revamped. The config file now accepts a
BuildmasterConfig['status'] entry, with a list of objects that perform status
delivery. The old config file entries which controlled the web status port
and the IRC bot have been deprecated in favor of adding instances to
['status']. The following status-delivery classes have been implemented, all
in the 'buildbot.status' package:

 client.PBListener(port, username, passwd)
 html.Waterfall(http_port, distrib_port)
 mail.MailNotifier(fromaddr, mode, extraRecipients..)
 words.IRC(host, nick, channels)

See the individual docstrings for details about how to use each one. You can
create new status-delivery objects by following the interfaces found in the
buildbot.interfaces module.


*** BuildFactory configuration process changed

The basic BuildFactory class is now defined in buildbot.process.factory
rather than buildbot.process.base, so you will have to update your config
files. factory.BuildFactory is the base class, which accepts a list of Steps
to run. See docs/factories.xhtml for details.

There are now easier-to-use BuildFactory classes for projects which use GNU
Autoconf, perl's MakeMaker (CPAN), python's distutils (but no unit tests),
and Twisted's Trial. Each one takes a separate 'source' Step to obtain the
source tree, and then fills in the rest of the Steps for you.


*** CVS/SVN VC steps unified, simplified

The confusing collection of arguments for the CVS step ('clobber=',
'copydir=', and 'export=') have been removed in favor of a single 'mode'
argument. This argument describes how you want to use the sources: whether
you want to update and compile everything in the same tree (mode='update'),
or do a fresh checkout and full build each time (mode='clobber'), or
something in between.

The SVN (Subversion) step has been unified and accepts the same mode=
parameter as CVS. New version control steps will obey the same interface.

Most of the old configuration arguments have been removed. You will need to
update your configuration files to use the new arguments. See
docs/steps.xhtml for a description of all the new parameters.


*** Preliminary Debian packaging added

Thanks to the contributions of Kirill Lapshin, we can now produce .deb
installer packages. These are still experimental, but they include init.d
startup/shutdown scripts, which the the new /usr/bin/buildbot to invoke
twistd. Create your buildmasters in /var/lib/buildbot/master/FOO, and your
slaves in /var/lib/buildbot/slave/BAR, then put FOO and BAR in the
appropriate places in /etc/default/buildbot . After that, the buildmasters
and slaves will be started at every boot.

Pre-built .debs are not yet distributed. Use 'debuild -uc -us' from the
source directory to create them.


** minor features


*** Source Stamps

Each build now has a "source stamp" which describes what sources it used. The
idea is that the sources for this particular build can be completely
regenerated from the stamp. The stamp is a tuple of (revision, patch), where
the revision depends on the VC system being used (for CVS it is either a
revision tag like "BUILDBOT-0_5_0" or a datestamp like "2004/07/23", for
Subversion it is a revision number like 11455). This must be combined with
information from the Builder that is constant across all builds (something to
point at the repository, and possibly a branch indicator for CVS and other VC
systems that don't fold this into the repository string).

The patch is an optional unified diff file, ready to be applied by running
'patch -p0 <PATCH' from inside the workdir. This provides support for the
'try' feature that will eventually allow developers to run buildbot tests on
their code before checking it in.


*** SIGHUP causes the buildmaster's configuration file to be re-read

*** IRC bot now has 'watch' command

You can now tell the buildbot's IRC bot to 'watch <buildername>' on a builder
which is currently performing a build. When that build is finished, the
buildbot will make an announcement (including the results of the build).

The IRC 'force build' command will also announce when the resulting build has
completed.


*** the 'force build' option on HTML and IRC status targets can be disabled

The html.Waterfall display and the words.IRC bot may be constructed with an
allowForce=False argument, which removes the ability to force a build through
these interfaces. Future versions will be able to restrict this build-forcing
capability to authenticated users. The per-builder HTML page no longer
displays the 'Force Build' buttons if it does not have this ability. Thanks
to Fred Drake for code and design suggestions.


*** master now takes 'projectName' and 'projectURL' settings

These strings allow the buildbot to describe what project it is working for.
At the moment they are only displayed on the Waterfall page, but in the next
release they will be retrieveable from the IRC bot as well.


*** survive recent (SVN) Twisted versions

The buildbot should run correctly (albeit with plenty of noisy deprecation
warnings) under the upcoming Twisted-2.0 release.


*** work-in-progress realtime Trial results acquisition

Jonathan Simms (<slyphon>) has been working on 'retrial', a rewrite of
Twisted's unit test framework that will most likely be available in
Twisted-2.0 . Although it is not yet complete, the buildbot will be able to
use retrial in such a way that build status is reported on a per-test basis,
in real time. This will be the beginning of fine-grained test tracking and
Problem management, described in docs/users.xhtml .