[Buildbot-devel] recent work: 'try'

[Brian Warner]

Hi all, I'm finally back from BlackHat/Defcon.. sorry for the radio silence
there, I'll try to catch up on mail as quickly as I can.

While on the road to Vegas, I was working on the much-touted 'try' feature,
attempting to honor my claim that it will *finally* make it into
0.7.0 . This has been on the "hey isn't this cool" list since the PyCon
presentation two and a half years ago, so I'm a bit embarrassed that I am
just now getting around to implementing it. Unfortunately there has been a
lot of support code that needed writing first.

I just merged the next-to-last bit of that code into sf.net CVS: the routine
that looks at your tree and figures out what revision it is based upon, and
extracts a patch that will allow the buildslaves to re-create your changes.
I've got this working for SVN and Darcs so far, and the Arch/Bazaar code
should work too (but I haven't finished the unit tests). CVS is going to be..
interesting.. I'm not yet sure how to make it work.

Once this getSourceStamp() functionality is done, the only remaining hard
part is how to securely get the baserev+patch from the developer to the
buildmaster. I'll include the draft chapter of the user's manual for this
feature below: I'm trying to figure out how to reduce the amount of
adminstrative overhead you (as the buildmaster admin) have to go through to
let your developers use this feature, while making absolutely sure that
non-developers can't submit arbitrary code to be run on the buildslaves. Let
me know what you think.

cheers,
 -Brian


8.2.3 try
---------

This lets a developer to ask the question "What would happen if I
committed this patch right now?". It runs the unit test suite (across
multiple build platforms) on the developer's current code, allowing
them to make sure they will not break the tree when they finally
commit their changes.

   The `buildbot try' command is meant to be run from within a
developer's local tree, and starts by figuring out the base revision
of that tree (what revision was current the last time the tree was
updated), and a patch that can be applied to that revision of the tree
to make it match the developer's copy. This (revision, patch) pair is
then sent to the buildmaster, which runs a build with that
SourceStamp. By default, the tool will emit status messages as the
builds run, and will not terminate until the first failure has been
detected (or the last success).

   For this command to work, several pieces must be in place:

TryScheduler
============

The buildmaster must have a `scheduler.TryScheduler' instance in the
config file's `c['schedulers']' list. This lets the administrator
control who may initiate these "trial" builds, which branches are
eligible for trial builds, and which Builders should be used for them.

   The `TryScheduler' has various means to accept build requests: all
of them enforce more security than the usual buildmaster ports do.
The source code patch that is provided with the trial build could be
used to compromise the buildslave accounts: the usual force-build
control channels can waste buildslave time but do not allow arbitrary
commands to be executed (since the code must be checked out from the
VC repository). The `TryScheduler' requires a bit more configuration
to insure that it cannot be abused in this way. There are currently
two ways to set this up:

_ssh_
     create a command queue maildir in the buildmaster's space, admin
     sets the ownership/permissions to only grant write access to
     trusted developers. 'buildbot try' formats an rfc822-style
     message (sourcestamp goes in a header, patch goes in the body)
     and adds it to the queuedir. The config file entries used by
     'buildbot try' either specify a local queuedir (for which write
     and mv are used) or a remote one (using scp and ssh). +: quite
     secure. -: requires fiddling outside the buildmaster config.

     To implement this, the buildslave invokes 'ssh -l username host
     buildbot tryserver ARGS', passing the patch contents over stdin.
     The arguments must include the inlet directory and the revision
     information.

_PB_
     each developer gets a username/passwd pair, known to the
     buildmaster.  'buildbot try' connects to the slaveport and
     identifies itself as that user, then sends a PB command to force
     the build (with the sourcestamp+diff as arguments). +: don't
     need a special group or filesystem fiddling. -: depending upon
     the Cred mechanics used, the password may be passed in plaintext
     or plaintext-equivalent. -: the username/passwd list must be
     maintained by the buildmaster admin, really you want it to
     remain equivalent to the VC system's list.


locating the master
===================

The `try' command needs to be told how to connect to the
`TryScheduler', and must know which of the authentication approaches
described above is in use by the buildmaster. You specify the
approach by using `--connect=ssh' or `--connect=pb' (or `try_connect
= 'ssh'' or `try_connect = 'pb'' in `.buildbot/options').

   For the PB approach, the command must be given a `--username' and
`--passwd' pair of arguments that match one of the entries in the
buildmaster's `tryusers' list. These arguments can also be provided
as `try_username' and `try_password' entries in the
`.buildbot/options' file.

   For the SSH approach, the command must be given `--tryhost',
`--username', and optionally `--password' (TODO: really?) to get to
the buildmaster host. It must also be given `--trydir', which points
to the inlet directory configured above. The trydir can be relative
to the user's home directory, but most of the time you will use an
explicit path like `~buildbot/project/trydir'. These arguments can be
provided in `.buildbot.options' as `try_host', `try_username',
`try_password', and `try_dir'.

specifying the VC system
========================

The `try' command also needs to know how to take the developer's
current tree and extract the (revision, patch) source-stamp pair.
Each VC system uses a different process, so you start by telling the
`try' command which VC system you are using, with an argument like
`--vc=cvs' or `--vc=tla'.

finding the top of the tree
===========================

Some VC systems (notably CVS and SVN) track each directory
more-or-less independently, which means the `try' command needs to
move up to the top of the project tree before it will be able to
construct a proper full-tree patch. To accomplish this, the `try'
command will crawl up through the parent directories until it finds a
marker file. The default name for this marker file is
`.buildbot-top', so when you are using CVS or SVN you should `touch
.buildbot-top' from the top of your tree before running `buildbot
try'. Alternatively, you can use a filename like `ChangeLog' or
`README', since many projects put one of these files in their
top-most directory (and nowhere else). To set this filename, use
`--try-topfile=ChangeLog', or set it in the options file with
`try_topfile = 'ChangeLog''.

   You can also manually set the top of the tree with
`--try-topdir=~/trees/mytree', or `try_topdir = '~/trees/mytree''. If
you use `try_topdir', in a `.buildbot/options' file, you will need a
separate options file for each tree you use, so it may be more
convenient to use the `try_topfile' approach instead.

   If the `try' command cannot find the top directory, it will abort
with an error message. Other VC systems which work on full projects
instead of individual directories (tla, baz, darcs, monotone) do not
require `try' to know the top directory, so the `--try-topfile' and
`--try-topdir' arguments will be ignored.

determining the revision and patch
==================================

Each VC system has a separate approach for determining the tree's base
revision and computing a patch.

`CVS'
     Wow, good question. We have to assume that you've done an `cvs
     update' on the whole tree... [TODO]

`SVN'
     `try' does a `svn status -u' to find the latest repository
     revision number (emitted on the last line in the "Status against
     revision: NN" message). It then performs an `svn diff -rNN' to
     find out how your tree differs from the repository version, and
     sends the resulting patch to the buildmaster. If your tree is not
     up to date, this will result in the "try" tree being created with
     the latest revision, then _backwards_ patches applied to bring it
     "back" to the version you actually checked out (plus your actual
     code changes), but this will still result in the correct tree
     being used for the build.

`baz'
     `try' does a `baz tree-id' to determine the fully-qualified
     version and patch identifier for the tree
     (ARCHIVE/VERSION-patch-NN), and uses the VERSION-patch-NN
     component as the base revision. It then does a `baz diff' to
     obtain the patch.

`tla'
     `try' does a `tla tree-version' to get the fully-qualified
     version identifier (ARCHIVE/VERSION), then takes the first line
     of `tla logs --reverse' to figure out the base revision. Then it
     does `tla changes --diffs' to obtain the patch.

`darcs'
     `darcs changes --context' emits a text file that contains a list
     of all patches back to and including the last tag was made. This
     text file (plus the location of a repository that contains all
     these patches) is sufficient to re-create the tree. Therefore
     the contents of this "context" file _are_ the revision stamp for
     a Darcs-controlled source tree.

     So `try' does a `darcs changes --context' to determine what your
     tree's base revision is, and then does a `darcs diff -u' to
     compute the patch relative to that revision.


waiting for results
===================

If you provide the `--wait' option (or `try_wait = True' in
`.buildbot/options'), the `buildbot try' command will wait until your
changes have either been proven good or bad before exiting. Unless
you use the `--quiet' option (or `try_quiet=True'), it will emit a
progress message every 60 seconds until the builds have completed.