[Buildbot-commits] buildbot/docs buildbot.texinfo,1.11,1.12
Brian Warner
warner at users.sourceforge.net
Wed Aug 3 23:47:30 UTC 2005
Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25404/docs
Modified Files:
buildbot.texinfo
Log Message:
Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-264
Creator: Brian Warner <warner at monolith.lothar.com>
start work on 'try', document the 'buildbot' command-line tool
* buildbot/trybuild.py: new file for 'try' utilities
(getSourceStamp): run in a tree, find out the baserev+patch
* buildbot/test/test_vc.py (VCBase.do_getpatch): test it,
implemented for SVN and Darcs, still working on Arch. I don't know
how to make CVS work yet.
* docs/buildbot.texinfo: document the 'buildbot' command-line
tool, including the not-yet-implemented 'try' feature, and the
in-flux .buildbot/ options directory.
Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.11
retrieving revision 1.12
diff -u -d -r1.11 -r1.12
--- buildbot.texinfo 20 Jul 2005 04:49:06 -0000 1.11
+++ buildbot.texinfo 3 Aug 2005 23:47:28 -0000 1.12
@@ -41,6 +41,7 @@
* Getting Source Code Changes:: Discovering when to run a build.
* Build Process:: Controlling how each build is run.
* Status Delivery:: Telling the world about the build's results.
+* Command-line tool::
* Resources:: Getting help.
* Developer's Appendix::
* Index:: Complete index.
@@ -175,6 +176,24 @@
* IRC Bot::
* PBListener::
+Command-line tool
+
+* Administrator Tools::
+* Developer Tools::
+* Other Tools::
+* .buildbot config directory::
+
+Developer Tools
+
+* statuslog::
+* statusgui::
+* try::
+
+Other Tools
+
+* sendchange::
+* debugclient::
+
@end detailmenu
@end menu
@@ -3336,7 +3355,7 @@
@end example
- at node Status Delivery, Resources, Build Process, Top
+ at node Status Delivery, Command-line tool, Build Process, Top
@chapter Status Delivery
More details are available in the docstrings for each class, use
@@ -3472,7 +3491,473 @@
status client can connect and retrieve status information.
@code{buildbot statusgui} is an example of such a status client.
- at node Resources, Developer's Appendix, Status Delivery, Top
+ at node Command-line tool, Resources, Status Delivery, Top
+ at chapter Command-line tool
+
+The @command{buildbot} command-line tool can be used to start or stop a
+buildmaster or buildbot, and to interact with a running buildmaster.
+Some of its subcommands are intended for buildmaster admins, while
+some are for developers who are editing the code that the buildbot is
+monitoring.
+
+ at menu
+* Administrator Tools::
+* Developer Tools::
+* Other Tools::
+* .buildbot config directory::
+ at end menu
+
+ at node Administrator Tools, Developer Tools, Command-line tool, Command-line tool
+ at section Administrator Tools
+
+The following @command{buildbot} sub-commands are intended for
+buildmaster administrators:
+
+ at heading master
+
+This creates a new directory and populates it with files that allow it
+to be used as a buildmaster's base directory.
+
+ at example
+buildbot master BASEDIR
+ at end example
+
+ at heading slave
+
+This creates a new directory and populates it with files that let it
+be used as a buildslave's base directory. You must provide several
+arguments, which are used to create the initial @file{buildbot.tac}
+file.
+
+ at example
+buildbot slave @var{BASEDIR} @var{MASTERHOST}:@var{PORT} @var{SLAVENAME} @var{PASSWORD}
+ at end example
+
+ at heading start
+
+This starts a buildmaster or buildslave which was already created in
+the given base directory. The daemon is launched in the background,
+with events logged to a file named @file{twistd.log}.
+
+ at example
+buildbot start BASEDIR
+ at end example
+
+ at heading stop
+
+This terminates the daemon (either buildmaster or buildslave) running
+in the given directory.
+
+ at example
+buildbot stop BASEDIR
+ at end example
+
+ at heading sighup
+
+This sends a SIGHUP to the buildmaster running in the given directory,
+which causes it to re-read its @file{master.cfg} file.
+
+ at example
+buildbot sighup BASEDIR
+ at end example
+
+ at node Developer Tools, Other Tools, Administrator Tools, Command-line tool
+ at section Developer Tools
+
+These tools are provided for use by the developers who are working on
+the code that the buildbot is monitoring.
+
+ at menu
+* statuslog::
+* statusgui::
+* try::
+ at end menu
+
+ at node statuslog, statusgui, Developer Tools, Developer Tools
+ at subsection statuslog
+
+ at example
+buildbot statuslog --master @var{MASTERHOST}:@var{PORT}
+ at end example
+
+This command starts a simple text-based status client, one which just
+prints out a new line each time an event occurs on the buildmaster.
+
+The @option{--master} option provides the location of the
+ at code{client.PBListener} status port, used to deliver build
+information to realtime status clients. The option is always in the
+form of a string, with hostname and port number separated by a colon
+(@code{HOSTNAME:PORTNUM}). Note that this port is @emph{not} the same
+as the slaveport (although a future version may allow the same port
+number to be used for both purposes).
+
+The @option{--master} option can also be provided by the
+ at code{masterstatus} name in @file{.buildbot/options} (@pxref{.buildbot
+config directory}).
+
+ at node statusgui, try, statuslog, Developer Tools
+ at subsection statusgui
+
+ at example
+buildbot statusgui --master @var{MASTERHOST}:@var{PORT}
+ at end example
+
+This command starts a simple Gtk+-based status client, which contains
+a few boxes for each Builder that change color as events occur. It
+uses the same @option{--master} argument as the @command{buildbot
+statuslog} command (@pxref{statuslog}).
+
+ at node try, , statusgui, Developer Tools
+ at subsection try
+
+This lets a developer to ask the question ``What would happen if I
+committed this patch right now?''. It runs the unit test suite (across
+multiple build platforms) on the developer's current code, allowing
+them to make sure they will not break the tree when they finally
+commit their changes.
+
+The @command{buildbot try} command is meant to be run from within a
+developer's local tree, and starts by figuring out the base revision
+of that tree (what revision was current the last time the tree was
+updated), and a patch that can be applied to that revision of the tree
+to make it match the developer's copy. This (revision, patch) pair is
+then sent to the buildmaster, which runs a build with that
+SourceStamp. By default, the tool will emit status messages as the
+builds run, and will not terminate until the first failure has been
+detected (or the last success).
+
+For this command to work, several pieces must be in place:
+
+ at heading TryScheduler
+
+The buildmaster must have a @code{scheduler.TryScheduler} instance in
+the config file's @code{c['schedulers']} list. This lets the
+administrator control who may initiate these ``trial'' builds, which
+branches are eligible for trial builds, and which Builders should be
+used for them.
+
+The @code{TryScheduler} has various means to accept build requests:
+all of them enforce more security than the usual buildmaster ports do.
+The source code patch that is provided with the trial build could be
+used to compromise the buildslave accounts: the usual force-build
+control channels can waste buildslave time but do not allow arbitrary
+commands to be executed (since the code must be checked out from the
+VC repository). The @code{TryScheduler} requires a bit more
+configuration to insure that it cannot be abused in this way. There
+are currently two ways to set this up:
+
+ at table @emph
+ at item ssh
+create a command queue maildir in the buildmaster's space, admin sets
+the ownership/permissions to only grant write access to trusted
+developers. 'buildbot try' formats an rfc822-style message
+(sourcestamp goes in a header, patch goes in the body) and adds it to
+the queuedir. The config file entries used by 'buildbot try' either
+specify a local queuedir (for which write and mv are used) or a remote
+one (using scp and ssh). +: quite secure. -: requires fiddling outside
+the buildmaster config.
+
+To implement this, the buildslave invokes 'ssh -l username host
+buildbot tryserver ARGS', passing the patch contents over stdin. The
+arguments must include the inlet directory and the revision
+information.
+
+ at item PB
+each developer gets a username/passwd pair, known to the buildmaster.
+'buildbot try' connects to the slaveport and identifies itself as that
+user, then sends a PB command to force the build (with the
+sourcestamp+diff as arguments). +: don't need a special group or
+filesystem fiddling. -: depending upon the Cred mechanics used, the
+password may be passed in plaintext or plaintext-equivalent. -: the
+username/passwd list must be maintained by the buildmaster admin,
+really you want it to remain equivalent to the VC system's list.
+
+ at end table
+
+ at heading locating the master
+
+The @command{try} command needs to be told how to connect to the
+ at code{TryScheduler}, and must know which of the authentication
+approaches described above is in use by the buildmaster. You specify
+the approach by using @option{--connect=ssh} or @option{--connect=pb}
+(or @code{try_connect = 'ssh'} or @code{try_connect = 'pb'} in
+ at file{.buildbot.options}).
+
+For the PB approach, the command must be given a @option{--username}
+and @option{--passwd} pair of arguments that match one of the entries
+in the buildmaster's @code{tryusers} list. These arguments can also be
+provided as @code{try_username} and @code{try_password} entries in the
+ at file{.buildbot/options} file.
+
+For the SSH approach, the command must be given @option{--tryhost},
+ at option{--username}, and optionally @option{--password} (TODO:
+really?) to get to the buildmaster host. It must also be given
+ at option{--trydir}, which points to the inlet directory configured
+above. The trydir can be relative to the user's home directory, but
+most of the time you will use an explicit path like
+ at file{~buildbot/project/trydir}. These arguments can be provided in
+ at file{.buildbot.options} as @code{try_host}, @code{try_username},
+ at code{try_password}, and @code{try_dir}.
+
+ at heading specifying the VC system
+
+The @command{try} command also needs to know how to take the
+developer's current tree and extract the (revision, patch)
+source-stamp pair. Each VC system uses a different process, so you
+start by telling the @command{try} command which VC system you are
+using, with an argument like @option{--vc=cvs} or @option{--vc=tla}.
+
+ at heading finding the top of the tree
+
+Some VC systems (notably CVS and SVN) track each directory
+more-or-less independently, which means the @command{try} command
+needs to move up to the top of the project tree before it will be able
+to construct a proper full-tree patch. To accomplish this, the
+ at command{try} command will crawl up through the parent directories
+until it finds a marker file. The default name for this marker file is
+ at file{.buildbot-top}, so when you are using CVS or SVN you should
+ at code{touch .buildbot-top} from the top of your tree before running
+ at command{buildbot try}. Alternatively, you can use a filename like
+ at file{ChangeLog} or @file{README}, since many projects put one of
+these files in their top-most directory (and nowhere else). To set
+this filename, use @option{--try-topfile=ChangeLog}, or set it in the
+options file with @code{try_topfile = 'ChangeLog'}.
+
+You can also manually set the top of the tree with
+ at option{--try-topdir=~/trees/mytree}, or @code{try_topdir =
+'~/trees/mytree'}. If you use @code{try_topdir}, in a
+ at file{.buildbot/options} file, you will need a separate options file
+for each tree you use, so it may be more convenient to use the
+ at code{try_topfile} approach instead.
+
+If the @command{try} command cannot find the top directory, it will
+abort with an error message. Other VC systems which work on full
+projects instead of individual directories (tla, baz, darcs, monotone)
+do not require @command{try} to know the top directory, so the
+ at option{--try-topfile} and @option{--try-topdir} arguments will be
+ignored.
+
+ at heading determining the revision and patch
+
+Each VC system has a separate approach for determining the tree's base
+revision and computing a patch.
+
+ at table @code
+
+ at item CVS
+Wow, good question. We have to assume that you've done an @code{cvs
+update} on the whole tree... [TODO]
+
+ at item SVN
+ at command{try} does a @code{svn status -u} to find the latest
+repository revision number (emitted on the last line in the ``Status
+against revision: NN'' message). It then performs an @code{svn diff
+-rNN} to find out how your tree differs from the repository version,
+and sends the resulting patch to the buildmaster. If your tree is not
+up to date, this will result in the ``try'' tree being created with
+the latest revision, then @emph{backwards} patches applied to bring it
+``back'' to the version you actually checked out (plus your actual
+code changes), but this will still result in the correct tree being
+used for the build.
+
+ at item baz
+ at command{try} does a @code{baz tree-id} to determine the
+fully-qualified version and patch identifier for the tree
+(ARCHIVE/VERSION--patch-NN), and uses the VERSION--patch-NN component
+as the base revision. It then does a @code{baz diff} to obtain the
+patch.
+
+ at item tla
+ at command{try} does a @code{tla tree-version} to get the
+fully-qualified version identifier (ARCHIVE/VERSION), then takes the
+first line of @code{tla logs --reverse} to figure out the base
+revision. Then it does @code{tla changes --diffs} to obtain the patch.
+
+ at item darcs
+ at code{darcs changes --context} emits a text file that contains a list
+of all patches back to and including the last tag was made. This text
+file (plus the location of a repository that contains all these
+patches) is sufficient to re-create the tree. Therefore the contents
+of this ``context'' file @emph{are} the revision stamp for a
+Darcs-controlled source tree.
+
+So @command{try} does a @code{darcs changes --context} to determine
+what your tree's base revision is, and then does a @code{darcs diff
+-u} to compute the patch relative to that revision.
+
+ at c TODO: monotone, git
+ at c TODO: new VC systems: mercurial (hg)
+ at end table
+
+ at heading waiting for results
+
+If you provide the @option{--wait} option (or @code{try_wait = True}
+in @file{.buildbot/options}), the @command{buildbot try} command will
+wait until your changes have either been proven good or bad before
+exiting. Unless you use the @option{--quiet} option (or
+ at code{try_quiet=True}), it will emit a progress message every 60
+seconds until the builds have completed.
+
+
+ at node Other Tools, .buildbot config directory, Developer Tools, Command-line tool
+ at section Other Tools
+
+These tools are generally used by buildmaster administrators.
+
+ at menu
+* sendchange::
+* debugclient::
+ at end menu
+
+ at node sendchange, debugclient, Other Tools, Other Tools
+ at subsection sendchange
+
+This command is used to tell the buildmaster about source changes. It
+is intended to be used from within a commit script, installed on the
+VC server.
+
+ at example
+buildbot sendchange --master @var{MASTERHOST}:@var{PORT} --username @var{USER} @var{FILENAMES..}
+ at end example
+
+There are other (optional) arguments which can influence the
+ at code{Change} that gets submitted:
+
+ at table @code
+ at item --revision_number
+This provides a (numeric) revision number for the change, used for VC systems
+that use numeric transaction numbers (like Subversion).
+
+ at item --revision
+This provides a (string) revision specifier, for VC systems that use
+strings (Arch would use something like patch-42, Darcs would use the
+patch name, etc).
+
+ at item --comments
+This provides the change comments as a single argument. You may want
+to use @option{--logfile} instead.
+
+ at item --logfile
+This instructs the tool to read the change comments from the given
+file. If you use @code{-} as the filename, the tool will read the
+change comments from stdin.
+ at end table
+
+
+ at node debugclient, , sendchange, Other Tools
+ at subsection debugclient
+
+ at example
+buildbot debugclient --master @var{MASTERHOST}:@var{PORT} --passwd @var{DEBUGPW}
+ at end example
+
+This launches a small Gtk+/Glade-based debug tool, connecting to the
+buildmaster's ``debug port''. This debug port shares the same port
+number as the slaveport (@pxref{Setting the slaveport}), but the
+ at code{debugPort} is only enabled if you set a debug password in the
+buildmaster's config file (@pxref{Debug options}). The
+ at option{--passwd} option must match the @code{c['debugPassword']}
+value.
+
+ at option{--master} can also be provided in @file{.debug/options} by the
+ at code{master} key. @option{--passwd} can be provided by the
+ at code{debugPassword} key.
+
+The @code{Connect} button must be pressed before any of the other
+buttons will be active. This establishes the connection to the
+buildmaster. The other sections of the tool are as follows:
+
+ at table @code
+ at item Reload .cfg
+Forces the buildmaster to reload its @file{master.cfg} file. This is
+equivalent to sending a SIGHUP to the buildmaster, but can be done
+remotely through the debug port. Note that it is a good idea to be
+watching the buildmaster's @file{twistd.log} as you reload the config
+file, as any errors which are detected in the config file will be
+announced there.
+
+ at item Rebuild .py
+(not yet implemented). The idea here is to use Twisted's ``rebuild''
+facilities to replace the buildmaster's running code with a new
+version. Even if this worked, it would only be used by buildbot
+developers.
+
+ at item poke IRC
+This locates a @code{words.IRC} status target and causes it to emit a
+message on all the channels to which it is currently connected. This
+was used to debug a problem in which the buildmaster lost the
+connection to the IRC server and did not attempt to reconnect.
+
+ at item Commit
+This allows you to inject a Change, just as if a real one had been
+delivered by whatever VC hook you are using. You can set the name of
+the committed file and the name of the user who is doing the commit.
+Optionally, you can also set a revision for the change. If the
+revision you provide looks like a number, it will be sent as an
+integer, otherwise it will be sent as a string.
+
+ at item Force Build
+This lets you force a Builder (selected by name) to start a build of
+the current source tree.
+
+ at item Currently
+(obsolete). This was used to manually set the status of the given
+Builder, but the status-assignment code was changed in an incompatible
+way and these buttons are no longer meaningful.
+
+ at end table
+
+
+ at node .buildbot config directory, , Other Tools, Command-line tool
+ at section .buildbot config directory
+
+Many of the @command{buildbot} tools must be told how to contact the
+buildmaster that they interact with. This specification can be
+provided as a command-line argument, but most of the time it will be
+easier to set them in an ``options'' file. The @command{buildbot}
+command will look for a special directory named @file{.buildbot},
+starting from the current directory (where the command was run) and
+crawling upwards, eventually looking in the user's home directory. It
+will look for a file named @file{options} in this directory, and will
+evaluate it as a python script, looking for certain names to be set.
+You can just put simple @code{name = 'value'} pairs in this file to
+set the options.
+
+For a description of the names used in this file, please see the
+documentation for the individual @command{buildbot} sub-commands. The
+following is a brief sample of what this file's contents could be.
+
+ at example
+# for status-reading tools
+masterstatus = 'buildbot.example.org:12345'
+# for 'sendchange' or the debug port
+master = 'buildbot.example.org:18990'
+debugPassword = 'eiv7Po'
+ at end example
+
+ at table @code
+ at item masterstatus
+Location of the @code{client.PBListener} status port, used by
+ at command{statuslog} and @command{statusgui}.
+
+ at item master
+Location of the @code{debugPort} (for @command{debugclient}). Also the
+location of the @code{pb.PBChangeSource} (for @command{sendchange}).
+Usually shares the slaveport, but a future version may make it
+possible to have these listen on a separate port number.
+
+ at item debugPassword
+Must match the value of @code{c['debugPassword']}, used to protect the
+debug port, for the @command{debugclient} command.
+
+ at item username
+Provides a default username for the @command{sendchange} command.
+
+ at end table
+
+
+
+ at node Resources, Developer's Appendix, Command-line tool, Top
@chapter Resources
The Buildbot's home page is at @uref{http://buildbot.sourceforge.net/}
More information about the Commits
mailing list