From warner at users.sourceforge.net Fri Jul 7 07:52:56 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Jul 2006 07:52:56 +0000 Subject: [Buildbot-commits] site ChangeLog,1.40,1.41 index.html,1.68,1.69 Message-ID: Update of /cvsroot/buildbot/site In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv23543 Modified Files: ChangeLog index.html Log Message: update ASCEND buildbot link Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/site/ChangeLog,v retrieving revision 1.40 retrieving revision 1.41 diff -u -d -r1.40 -r1.41 --- ChangeLog 5 Jun 2006 00:47:49 -0000 1.40 +++ ChangeLog 7 Jul 2006 07:52:54 -0000 1.41 @@ -1,3 +1,7 @@ +2006-07-07 Brian Warner + + * index.html: update ASCEND buildbot link + 2006-06-04 Brian Warner * index.html: add the Auger waterfall too Index: index.html =================================================================== RCS file: /cvsroot/buildbot/site/index.html,v retrieving revision 1.68 retrieving revision 1.69 diff -u -d -r1.68 -r1.69 --- index.html 5 Jun 2006 00:47:49 -0000 1.68 +++ index.html 7 Jul 2006 07:52:54 -0000 1.69 @@ -425,7 +425,7 @@ ASCEND - + Buildbot @@ -572,5 +572,5 @@ href="http://creativecommons.org/licenses/by-sa/2.5/">Creative Commons Attribution Share-Alike license.

-Last modified: Sun Jun 4 17:44:56 PDT 2006 +Last modified: Fri Jul 7 00:52:00 PDT 2006 From warner at users.sourceforge.net Wed Jul 12 21:36:35 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Wed, 12 Jul 2006 21:36:35 +0000 Subject: [Buildbot-commits] site ChangeLog,1.41,1.42 index.html,1.69,1.70 Message-ID: Update of /cvsroot/buildbot/site In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16384 Modified Files: ChangeLog index.html Log Message: ethereal became wireshark, fix Browse CVS link Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/site/ChangeLog,v retrieving revision 1.41 retrieving revision 1.42 diff -u -d -r1.41 -r1.42 --- ChangeLog 7 Jul 2006 07:52:54 -0000 1.41 +++ ChangeLog 12 Jul 2006 21:36:32 -0000 1.42 @@ -1,3 +1,7 @@ +2006-07-12 Brian Warner + + * index.html: s/Ethereal/Wireshark/g, fix Browse-CVS link. + 2006-07-07 Brian Warner * index.html: update ASCEND buildbot link Index: index.html =================================================================== RCS file: /cvsroot/buildbot/site/index.html,v retrieving revision 1.69 retrieving revision 1.70 diff -u -d -r1.69 -r1.70 --- index.html 7 Jul 2006 07:52:54 -0000 1.69 +++ index.html 12 Jul 2006 21:36:32 -0000 1.70 @@ -51,7 +51,7 @@ href="http://www.lothar.com/warner-gpg.html">here.
  • The latest code is available from CVS for browsing + href="http://buildbot.cvs.sourceforge.net/buildbot/buildbot/">browsing or read-only checkout. There are also Arch and Darcs repositories which track the main CVS tree and provide lower-latency access than anonymous CVS, details @@ -319,7 +319,9 @@ Ulf Lamping reports that the Ethereal project, a well-known packet sniffer, uses a buildbot for their cross-platform compatibility - testing. + testing. Joerg Mayer reports that they've changed their name to Wireshark and their new buildbot + is here. @@ -572,5 +574,5 @@ href="http://creativecommons.org/licenses/by-sa/2.5/">Creative Commons Attribution Share-Alike license.

    -Last modified: Fri Jul 7 00:52:00 PDT 2006 +Last modified: Wed Jul 12 14:32:41 PDT 2006 From warner at users.sourceforge.net Sat Jul 15 08:50:37 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 15 Jul 2006 08:50:37 +0000 Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.43,1.44 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv22884/buildbot Modified Files: interfaces.py Log Message: [project @ hush t.p.components.Interface deprecation warnings by using twcompat] Original author: warner at lothar.com Date: 2006-06-29 17:49:02 Index: interfaces.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v retrieving revision 1.43 retrieving revision 1.44 diff -u -d -r1.43 -r1.44 --- interfaces.py 15 Jun 2006 05:47:13 -0000 1.43 +++ interfaces.py 15 Jul 2006 08:50:35 -0000 1.44 @@ -5,7 +5,7 @@ Define the interfaces that are implemented by various buildbot classes. """ -from twisted.python.components import Interface +from buildbot.twcompat import Interface # exceptions that can be raised while trying to start a build class NoSlaveError(Exception): From warner at users.sourceforge.net Sat Jul 15 08:50:37 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 15 Jul 2006 08:50:37 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.680,1.681 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv22884 Modified Files: ChangeLog Log Message: [project @ hush t.p.components.Interface deprecation warnings by using twcompat] Original author: warner at lothar.com Date: 2006-06-29 17:49:02 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.680 retrieving revision 1.681 diff -u -d -r1.680 -r1.681 --- ChangeLog 28 Jun 2006 20:05:45 -0000 1.680 +++ ChangeLog 15 Jul 2006 08:50:34 -0000 1.681 @@ -1,3 +1,11 @@ +2006-06-29 Brian Warner + + * buildbot/interfaces.py: get Interface from b.twcompat to hush + deprecation warnings under newer Twisteds (by using + zope.interface.Interface instead of old twisted.python.components + stuff) + * buildbot/slave/interfaces.py: same + 2006-06-28 Brian Warner * buildbot/slave/commands.py (SVN): add --non-interactive to all From warner at users.sourceforge.net Sat Jul 15 08:50:37 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 15 Jul 2006 08:50:37 +0000 Subject: [Buildbot-commits] buildbot/buildbot/slave interfaces.py,1.1,1.2 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/slave In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv22884/buildbot/slave Modified Files: interfaces.py Log Message: [project @ hush t.p.components.Interface deprecation warnings by using twcompat] Original author: warner at lothar.com Date: 2006-06-29 17:49:02 Index: interfaces.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/interfaces.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- interfaces.py 3 Dec 2004 22:54:52 -0000 1.1 +++ interfaces.py 15 Jul 2006 08:50:35 -0000 1.2 @@ -1,6 +1,6 @@ #! /usr/bin/python -from twisted.python.components import Interface +from buildbot.twcompat import Interface class ISlaveCommand(Interface): """This interface is implemented by all of the buildslave's Command From warner at users.sourceforge.net Sun Jul 16 00:58:33 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:58:33 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.681,1.682 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7415 Modified Files: ChangeLog Log Message: [project @ fix some epydoc issues] Original author: warner at lothar.com Date: 2006-07-16 00:03:55 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.681 retrieving revision 1.682 diff -u -d -r1.681 -r1.682 --- ChangeLog 15 Jul 2006 08:50:34 -0000 1.681 +++ ChangeLog 16 Jul 2006 00:58:31 -0000 1.682 @@ -1,3 +1,8 @@ +2006-07-15 Brian Warner + + * buildbot/twcompat.py (_which): fix some epydoc issues + * buildbot/status/html.py (Waterfall.__init__): same + 2006-06-29 Brian Warner * buildbot/interfaces.py: get Interface from b.twcompat to hush From warner at users.sourceforge.net Sun Jul 16 00:58:33 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:58:33 +0000 Subject: [Buildbot-commits] buildbot/buildbot twcompat.py,1.5,1.6 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7415/buildbot Modified Files: twcompat.py Log Message: [project @ fix some epydoc issues] Original author: warner at lothar.com Date: 2006-07-16 00:03:55 Index: twcompat.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/twcompat.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- twcompat.py 24 Apr 2006 09:03:07 -0000 1.5 +++ twcompat.py 16 Jul 2006 00:58:31 -0000 1.6 @@ -264,7 +264,7 @@ @param flags: Arguments to L{os.access}. @rtype: C{list} - @param: A list of the full paths to files found, in the + @return: A list of the full paths to files found, in the order in which they were found. """ result = [] From warner at users.sourceforge.net Sun Jul 16 00:58:33 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:58:33 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.86,1.87 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7415/buildbot/status Modified Files: html.py Log Message: [project @ fix some epydoc issues] Original author: warner at lothar.com Date: 2006-07-16 00:03:55 Index: html.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v retrieving revision 1.86 retrieving revision 1.87 diff -u -d -r1.86 -r1.87 --- html.py 3 Jun 2006 20:21:04 -0000 1.86 +++ html.py 16 Jul 2006 00:58:31 -0000 1.87 @@ -1695,7 +1695,7 @@ Many search engines request this file and obey the rules in it. E.g. to disallow them to crawl the status page, put the following two lines in - robots.txt: + robots.txt:: User-agent: * Disallow: / """ From warner at users.sourceforge.net Sun Jul 16 00:59:07 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:59:07 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.682,1.683 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7652 Modified Files: ChangeLog Log Message: [project @ implement manhole (ssh-based, colorized/line-editing terminal)] Original author: warner at lothar.com Date: 2006-07-16 00:55:25 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.682 retrieving revision 1.683 diff -u -d -r1.682 -r1.683 --- ChangeLog 16 Jul 2006 00:58:31 -0000 1.682 +++ ChangeLog 16 Jul 2006 00:59:05 -0000 1.683 @@ -1,5 +1,15 @@ 2006-07-15 Brian Warner + * buildbot/manhole.py: move all Manhole-related code out to this + module. Implement SSH-based manholes (with TwistedConch), and move + to conch's nifty line-editing syntax-coloring REPL shell instead + of the boring non-editing monochromatic (and deprecated) old + 'telnet' protocol. + * buildbot/master.py: remove all Manhole-related code + (BuildMaster.loadConfig._add): make sure the old manhole is + removed before we add the new one + * docs/buildbot.texinfo (Debug options): document new Manhole options + * buildbot/twcompat.py (_which): fix some epydoc issues * buildbot/status/html.py (Waterfall.__init__): same From warner at users.sourceforge.net Sun Jul 16 00:59:07 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:59:07 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.57,1.58 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7652/docs Modified Files: buildbot.texinfo Log Message: [project @ implement manhole (ssh-based, colorized/line-editing terminal)] Original author: warner at lothar.com Date: 2006-07-16 00:55:25 Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.57 retrieving revision 1.58 diff -u -d -r1.57 -r1.58 --- buildbot.texinfo 20 Jun 2006 08:08:58 -0000 1.57 +++ buildbot.texinfo 16 Jul 2006 00:59:05 -0000 1.58 @@ -2162,13 +2162,49 @@ @end example @cindex c['manhole'] -If you set @code{c['manhole']} to an instance of the - at code{buildbot.master.Manhole} class, you can telnet into the -buildmaster and get an interactive Python shell, which may be useful -for debugging buildbot internals. It is probably only useful for -buildbot developers. It exposes full access to the buildmaster's -account (including the ability to modify and delete files), so it -should not be enabled with a weak or easily guessable password. +If you set @code{c['manhole']} to an instance of one of the classes in + at code{buildbot.manhole}, you can telnet or ssh into the buildmaster +and get an interactive Python shell, which may be useful for debugging +buildbot internals. It is probably only useful for buildbot +developers. It exposes full access to the buildmaster's account +(including the ability to modify and delete files), so it should not +be enabled with a weak or easily guessable password. + +There are three separate @code{Manhole} classes. Two of them use SSH, +one uses unencrypted telnet. Two of them use a username+password +combination to grant access, one of them uses an SSH-style + at file{authorized_keys} file which contains a list of ssh public keys. + + at table @code + at item manhole.AuthorizedKeysManhole +You construct this with the name of a file that contains one SSH +public key per line, just like @file{~/.ssh/authorized_keys}. If you +provide a non-absolute filename, it will be interpreted relative to +the buildmaster's base directory. + + at item manhole.PasswordManhole +This one accepts SSH connections but asks for a username and password +when authenticating. It accepts only one such pair. + + + at item manhole.TelnetManhole +This accepts regular unencrypted telnet connections, and asks for a +username/password pair before providing access. Because this +username/password is transmitted in the clear, and because Manhole +access to the buildmaster is equivalent to granting full shell +privileges to both the buildmaster and all the buildslaves (and to all +accounts which then run code produced by the buildslaves), it is +highly recommended that you use one of the SSH manholes instead. + + at end table + + at example +# some examples: +from buildbot import manhole +c['manhole'] = manhole.AuthorizedKeysManhole(1234, "authorized_keys") +c['manhole'] = manhole.PasswordManhole(1234, "alice", "mysecretpassword") +c['manhole'] = manhole.TelnetManhole(1234, "bob", "snoop_my_password_please") + at end example The @code{Manhole} instance can be configured to listen on a specific port. You may wish to have this listening port bind to the loopback @@ -2176,15 +2212,18 @@ restrict access to clients which are running on the same host. @example -from buildbot.master import Manhole -c['manhole'] = Manhole("tcp:9999:interface=127.0.0.1", "admin", "password") +from buildbot.manhole import PasswordManhole +c['manhole'] = PasswordManhole("tcp:9999:interface=127.0.0.1","admin","passwd") @end example To have the @code{Manhole} listen on all interfaces, use - at code{"tcp:9999"}. This port specification uses + at code{"tcp:9999"} or simply 9999. This port specification uses @code{twisted.application.strports}, so you can make it listen on SSL or even UNIX-domain sockets if you want. +Note that using any Manhole requires that the TwistedConch package be +installed, and that you be using Twisted version 2.0 or later. + @node Getting Source Code Changes, Build Process, Configuration, Top @chapter Getting Source Code Changes From warner at users.sourceforge.net Sun Jul 16 00:59:07 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:59:07 +0000 Subject: [Buildbot-commits] buildbot/buildbot manhole.py, NONE, 1.1 master.py, 1.92, 1.93 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7652/buildbot Modified Files: master.py Added Files: manhole.py Log Message: [project @ implement manhole (ssh-based, colorized/line-editing terminal)] Original author: warner at lothar.com Date: 2006-07-16 00:55:25 --- NEW FILE: manhole.py --- import os.path import binascii, base64 from twisted.python import log from twisted.application import service, strports from twisted.cred import checkers, portal from twisted.conch import manhole, telnet, manhole_ssh, checkers as conchc from twisted.conch.insults import insults from twisted.internet import protocol from buildbot.util import ComparableMixin from zope.interface import implements # requires Twisted-2.0 or later # makeTelnetProtocol and _TelnetRealm are for the TelnetManhole class makeTelnetProtocol: # this curries the 'portal' argument into a later call to # TelnetTransport() def __init__(self, portal): self.portal = portal def __call__(self): auth = telnet.AuthenticatingTelnetProtocol return telnet.TelnetTransport(auth, self.portal) class _TelnetRealm: implements(portal.IRealm) def __init__(self, namespace_maker): self.namespace_maker = namespace_maker def requestAvatar(self, avatarId, *interfaces): if telnet.ITelnetProtocol in interfaces: namespace = self.namespace_maker() p = telnet.TelnetBootstrapProtocol(insults.ServerProtocol, manhole.ColoredManhole, namespace) return (telnet.ITelnetProtocol, p, lambda: None) raise NotImplementedError() class chainedProtocolFactory: # this curries the 'namespace' argument into a later call to # chainedProtocolFactory() def __init__(self, namespace): self.namespace = namespace def __call__(self): return insults.ServerProtocol(manhole.ColoredManhole, self.namespace) class AuthorizedKeysChecker(conchc.SSHPublicKeyDatabase): """Accept connections using SSH keys from a given file. SSHPublicKeyDatabase takes the username that the prospective client has requested and attempts to get a ~/.ssh/authorized_keys file for that username. This requires root access, so it isn't as useful as you'd like. Instead, this subclass looks for keys in a single file, given as an argument. This file is typically kept in the buildmaster's basedir. The file should have 'ssh-dss ....' lines in it, just like authorized_keys. """ def __init__(self, authorized_keys_file): self.authorized_keys_file = os.path.expanduser(authorized_keys_file) def checkKey(self, credentials): f = open(self.authorized_keys_file) for l in f.readlines(): l2 = l.split() if len(l2) < 2: continue try: if base64.decodestring(l2[1]) == credentials.blob: return 1 except binascii.Error: continue return 0 class _BaseManhole(service.MultiService): """This provides remote access to a python interpreter (a read/exec/print loop) embedded in the buildmaster via an internal SSH server. This allows detailed inspection of the buildmaster state. It is of most use to buildbot developers. Connect to this by running an ssh client. """ def __init__(self, port, checker, using_ssh=True): """ @type port: string or int @param port: what port should the Manhole listen on? This is a strports specification string, like 'tcp:12345' or 'tcp:12345:interface=127.0.0.1'. Bare integers are treated as a simple tcp port. @type checker: an object providing the L{twisted.cred.checkers.ICredentialsChecker} interface @param checker: if provided, this checker is used to authenticate the client instead of using the username/password scheme. You must either provide a username/password or a Checker. Some useful values are:: import twisted.cred.checkers as credc import twisted.conch.checkers as conchc c = credc.AllowAnonymousAccess # completely open c = credc.FilePasswordDB(passwd_filename) # file of name:passwd c = conchc.UNIXPasswordDatabase # getpwnam() (probably /etc/passwd) @type using_ssh: bool @param using_ssh: If True, accept SSH connections. If False, accept regular unencrypted telnet connections. """ # unfortunately, these don't work unless we're running as root #c = credc.PluggableAuthenticationModulesChecker: PAM #c = conchc.SSHPublicKeyDatabase() # ~/.ssh/authorized_keys # and I can't get UNIXPasswordDatabase to work service.MultiService.__init__(self) if type(port) is int: port = "tcp:%d" % port self.port = port # for comparison later self.checker = checker # to maybe compare later def makeNamespace(): # close over 'self' so we can get access to .parent later master = self.parent namespace = { 'master': master, 'status': master.getStatus(), } return namespace def makeProtocol(): namespace = makeNamespace() p = insults.ServerProtocol(manhole.ColoredManhole, namespace) return p self.using_ssh = using_ssh if using_ssh: r = manhole_ssh.TerminalRealm() r.chainedProtocolFactory = makeProtocol p = portal.Portal(r, [self.checker]) f = manhole_ssh.ConchFactory(p) else: r = _TelnetRealm(makeNamespace) p = portal.Portal(r, [self.checker]) f = protocol.ServerFactory() f.protocol = makeTelnetProtocol(p) s = strports.service(self.port, f) s.setServiceParent(self) def startService(self): service.MultiService.startService(self) if self.using_ssh: via = "via SSH" else: via = "via telnet" log.msg("Manhole listening %s on port %s" % (via, self.port)) class TelnetManhole(_BaseManhole, ComparableMixin): """This Manhole accepts unencrypted (telnet) connections, and requires a username and password authorize access. You are encouraged to use the encrypted ssh-based manhole classes instead.""" compare_attrs = ["port", "username", "password"] def __init__(self, port, username, password): """ @type port: string or int @param port: what port should the Manhole listen on? This is a strports specification string, like 'tcp:12345' or 'tcp:12345:interface=127.0.0.1'. Bare integers are treated as a simple tcp port. @param username: @param password: username= and password= form a pair of strings to use when authenticating the remote user. """ self.username = username self.password = password c = checkers.InMemoryUsernamePasswordDatabaseDontUse() c.addUser(username, password) _BaseManhole.__init__(self, port, c, using_ssh=False) class PasswordManhole(_BaseManhole, ComparableMixin): """This Manhole accepts encrypted (ssh) connections, and requires a username and password to authorize access. """ compare_attrs = ["port", "username", "password"] def __init__(self, port, username, password): """ @type port: string or int @param port: what port should the Manhole listen on? This is a strports specification string, like 'tcp:12345' or 'tcp:12345:interface=127.0.0.1'. Bare integers are treated as a simple tcp port. @param username: @param password: username= and password= form a pair of strings to use when authenticating the remote user. """ self.username = username self.password = password c = checkers.InMemoryUsernamePasswordDatabaseDontUse() c.addUser(username, password) _BaseManhole.__init__(self, port, c) class AuthorizedKeysManhole(_BaseManhole, ComparableMixin): """This Manhole accepts ssh connections, and requires that the prospective client have an ssh private key that matches one of the public keys in our authorized_keys file. It is created with the name of a file that contains the public keys that we will accept.""" compare_attrs = ["port", "keyfile"] def __init__(self, port, keyfile): """ @type port: string or int @param port: what port should the Manhole listen on? This is a strports specification string, like 'tcp:12345' or 'tcp:12345:interface=127.0.0.1'. Bare integers are treated as a simple tcp port. @param keyfile: the name of a file (relative to the buildmaster's basedir) that contains SSH public keys of authorized users, one per line. This is the exact same format as used by sshd in ~/.ssh/authorized_keys . """ # TODO: expanduser this, and make it relative to the buildmaster's # basedir self.keyfile = keyfile c = AuthorizedKeysChecker(keyfile) _BaseManhole.__init__(self, port, c) class ArbitraryCheckerManhole(_BaseManhole, ComparableMixin): """This Manhole accepts ssh connections, but uses an arbitrary user-supplied 'checker' object to perform authentication.""" compare_attrs = ["port", "checker"] def __init__(self, port, checker): """ @type port: string or int @param port: what port should the Manhole listen on? This is a strports specification string, like 'tcp:12345' or 'tcp:12345:interface=127.0.0.1'. Bare integers are treated as a simple tcp port. @param checker: an instance of a twisted.cred 'checker' which will perform authentication """ _BaseManhole.__init__(self, port, checker) Index: master.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/master.py,v retrieving revision 1.92 retrieving revision 1.93 diff -u -d -r1.92 -r1.93 --- master.py 17 Apr 2006 19:22:34 -0000 1.92 +++ master.py 16 Jul 2006 00:59:05 -0000 1.93 @@ -17,7 +17,6 @@ from twisted.cred import portal, checkers from twisted.application import service, strports from twisted.persisted import styles -from twisted.manhole import telnet # sibling imports from buildbot import util @@ -424,28 +423,7 @@ ######################################## -class Manhole(service.MultiService, util.ComparableMixin): - compare_attrs = ["port", "username", "password"] - def __init__(self, port, username, password): - service.MultiService.__init__(self) - if type(port) is int: - port = "tcp:%d" % port - self.port = port - self.username = username - self.password = password - self.f = f = telnet.ShellFactory() - f.username = username - f.password = password - s = strports.service(port, f) - s.setServiceParent(self) - - def startService(self): - log.msg("Manhole listening on port %s" % self.port) - service.MultiService.startService(self) - master = self.parent - self.f.namespace['master'] = master - self.f.namespace['status'] = master.getStatus() class DebugPerspective(NewCredPerspective): def attached(self, mind): @@ -842,10 +820,15 @@ if self.manhole: # disownServiceParent may return a Deferred d.addCallback(lambda res: self.manhole.disownServiceParent()) - self.manhole = None + def _remove(res): + self.manhole = None + return res + d.addCallback(_remove) if manhole: - self.manhole = manhole - manhole.setServiceParent(self) + def _add(res): + self.manhole = manhole + manhole.setServiceParent(self) + d.addCallback(_add) # add/remove self.botmaster.builders to match builders. The # botmaster will handle startup/shutdown issues. From warner at users.sourceforge.net Sun Jul 16 00:59:16 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:59:16 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.683,1.684 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7834 Modified Files: ChangeLog Log Message: [project @ twcompat.py: fix minor typo] Original author: warner at lothar.com Date: 2006-07-16 00:57:16 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.683 retrieving revision 1.684 diff -u -d -r1.683 -r1.684 --- ChangeLog 16 Jul 2006 00:59:05 -0000 1.683 +++ ChangeLog 16 Jul 2006 00:59:14 -0000 1.684 @@ -1,5 +1,7 @@ 2006-07-15 Brian Warner + * buildbot/twcompat.py: fix minor typo in comments + * buildbot/manhole.py: move all Manhole-related code out to this module. Implement SSH-based manholes (with TwistedConch), and move to conch's nifty line-editing syntax-coloring REPL shell instead From warner at users.sourceforge.net Sun Jul 16 00:59:16 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 00:59:16 +0000 Subject: [Buildbot-commits] buildbot/buildbot twcompat.py,1.6,1.7 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7834/buildbot Modified Files: twcompat.py Log Message: [project @ twcompat.py: fix minor typo] Original author: warner at lothar.com Date: 2006-07-16 00:57:16 Index: twcompat.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/twcompat.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- twcompat.py 16 Jul 2006 00:58:31 -0000 1.6 +++ twcompat.py 16 Jul 2006 00:59:14 -0000 1.7 @@ -7,7 +7,7 @@ implements. Use this like the following. -from buildbot.tcompat import implements +from buildbot.twcompat import implements class Foo: if implements: implements(IFoo) From warner at users.sourceforge.net Sun Jul 16 03:34:07 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 03:34:07 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.684,1.685 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv3223 Modified Files: ChangeLog Log Message: [project @ rearrange sample.cfg into major sections, update manhole example] Original author: warner at lothar.com Date: 2006-07-16 03:31:00 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.684 retrieving revision 1.685 diff -u -d -r1.684 -r1.685 --- ChangeLog 16 Jul 2006 00:59:14 -0000 1.684 +++ ChangeLog 16 Jul 2006 03:34:05 -0000 1.685 @@ -1,5 +1,8 @@ 2006-07-15 Brian Warner + * buildbot/scripts/sample.cfg: update manhole example, arrange into + major sections + * buildbot/twcompat.py: fix minor typo in comments * buildbot/manhole.py: move all Manhole-related code out to this From warner at users.sourceforge.net Sun Jul 16 03:34:07 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 03:34:07 +0000 Subject: [Buildbot-commits] buildbot/buildbot/scripts sample.cfg,1.9,1.10 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/scripts In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv3223/buildbot/scripts Modified Files: sample.cfg Log Message: [project @ rearrange sample.cfg into major sections, update manhole example] Original author: warner at lothar.com Date: 2006-07-16 03:31:00 Index: sample.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/sample.cfg,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- sample.cfg 29 May 2006 00:10:26 -0000 1.9 +++ sample.cfg 16 Jul 2006 03:34:05 -0000 1.10 @@ -9,23 +9,27 @@ # dictionary has a variety of keys to control different aspects of the # buildmaster. They are documented in docs/config.xhtml . -import os.path -from buildbot.changes.freshcvs import FreshCVSSource -from buildbot.scheduler import Scheduler -from buildbot.process import step, factory -from buildbot.status import html -s = factory.s # This is the dictionary that the buildmaster pays attention to. We also use # a shorter alias to save typing. c = BuildmasterConfig = {} +####### BUILDSLAVES + # the 'bots' list defines the set of allowable buildslaves. Each element is a # tuple of bot-name and bot-password. These correspond to values given to the # buildslave's mktap invocation. c['bots'] = [("bot1name", "bot1passwd")] +# 'slavePortnum' defines the TCP port to listen on. This must match the value +# configured into the buildslaves (with their --master option) + +c['slavePortnum'] = 9989 + + +####### CHANGESOURCES + # the 'sources' list tells the buildmaster how it should find out about # source code changes. Any class which implements IChangeSource can be added # to this list: there are several in buildbot/changes/*.py to choose from. @@ -41,6 +45,7 @@ # then you could use the following buildmaster Change Source to subscribe to # the FreshCVS daemon and be notified on every commit: # +#from buildbot.changes.freshcvs import FreshCVSSource #fc_source = FreshCVSSource("cvs.example.com", 4519, "foo", "bar") #c['sources'].append(fc_source) @@ -52,14 +57,18 @@ #c['sources'].append(PBChangeSource()) +####### SCHEDULERS + ## configure the Schedulers +from buildbot.scheduler import Scheduler c['schedulers'] = [] c['schedulers'].append(Scheduler(name="all", branch=None, treeStableTimer=2*60, builderNames=["buildbot-full"])) +####### BUILDERS # the 'builders' list defines the Builders. Each one is configured with a # dictionary, using the following keys: @@ -84,6 +93,8 @@ builders = [] +from buildbot.process import step, factory +s = factory.s source = s(step.CVS, cvsroot=cvsroot, cvsmodule=cvsmodule, login="", mode="copy") f1 = factory.Trial(source, tests="buildbot.test", @@ -97,16 +108,16 @@ } c['builders'] = [b1] -# 'slavePortnum' defines the TCP port to listen on. This must match the value -# configured into the buildslaves (with their --master option) -c['slavePortnum'] = 9989 +####### STATUS TARGETS # 'status' is a list of Status Targets. The results of each build will be # pushed to these targets. buildbot/status/*.py has a variety to choose from, # including web pages, email senders, and IRC bots. c['status'] = [] + +from buildbot.status import html c['status'].append(html.Waterfall(http_port=8010)) # from buildbot.status import mail @@ -118,6 +129,8 @@ # channels=["#example"])) +####### DEBUGGING OPTIONS + # if you set 'debugPassword', then you can connect to the buildmaster with # the diagnostic tool in contrib/debugclient.py . From this tool, you can # manually force builds and inject changes, which may be useful for testing @@ -125,13 +138,18 @@ # before you have a functioning 'sources' set up). The debug tool uses the # same port number as the slaves do: 'slavePortnum'. -c['debugPassword'] = "debugpassword" +#c['debugPassword'] = "debugpassword" -# if you set 'manhole', you can telnet into the buildmaster and get an +# if you set 'manhole', you can ssh into the buildmaster and get an # interactive python shell, which may be useful for debugging buildbot -# internals. It is probably only useful for buildbot developers. -#from buildbot.master import Manhole -#c['manhole'] = Manhole("tcp:9999:interface=127.0.0.1", "admin", "password") +# internals. It is probably only useful for buildbot developers. You can also +# use an authorized_keys file, or plain telnet. +#from buildbot import manhole +#c['manhole'] = manhole.PasswordManhole("tcp:9999:interface=127.0.0.1", +# "admin", "password") + + +####### PROJECT IDENTITY # the 'projectName' string will be used to describe the project that this # buildbot is working on. For example, it is used as the title of the From warner at users.sourceforge.net Sun Jul 16 03:34:15 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 03:34:15 +0000 Subject: [Buildbot-commits] buildbot ChangeLog, 1.685, 1.686 setup.py, 1.37, 1.38 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv3446 Modified Files: ChangeLog setup.py Log Message: [project @ setup.py: minor comment] Original author: warner at lothar.com Date: 2006-07-16 03:32:51 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.685 retrieving revision 1.686 diff -u -d -r1.685 -r1.686 --- ChangeLog 16 Jul 2006 03:34:05 -0000 1.685 +++ ChangeLog 16 Jul 2006 03:34:13 -0000 1.686 @@ -1,5 +1,8 @@ 2006-07-15 Brian Warner + * setup.py: minor comment.. does the classifiers= argument prevent + the setup.py script from working on python2.2/2.3? + * buildbot/scripts/sample.cfg: update manhole example, arrange into major sections Index: setup.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/setup.py,v retrieving revision 1.37 retrieving revision 1.38 diff -u -d -r1.37 -r1.38 --- setup.py 12 Jun 2006 08:35:53 -0000 1.37 +++ setup.py 16 Jul 2006 03:34:13 -0000 1.38 @@ -45,6 +45,7 @@ author_email="warner-buildbot at lothar.com", url="http://buildbot.sourceforge.net/", license="GNU GPL", + # does this classifiers= mean that this can't be installed on 2.2/2.3? classifiers=[ 'Development Status :: 4 - Beta', 'Environment :: No Input/Output (Daemon)', From warner at users.sourceforge.net Sun Jul 16 03:34:23 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Jul 2006 03:34:23 +0000 Subject: [Buildbot-commits] buildbot ChangeLog, 1.686, 1.687 Makefile, 1.18, 1.19 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv3483 Modified Files: ChangeLog Makefile Log Message: [project @ Makefile: add some-apidocs target] Original author: warner at lothar.com Date: 2006-07-16 03:33:39 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.686 retrieving revision 1.687 diff -u -d -r1.686 -r1.687 --- ChangeLog 16 Jul 2006 03:34:13 -0000 1.686 +++ ChangeLog 16 Jul 2006 03:34:21 -0000 1.687 @@ -1,5 +1,7 @@ 2006-07-15 Brian Warner + * Makefile (some-apidocs): new target to build only some epydocs + * setup.py: minor comment.. does the classifiers= argument prevent the setup.py script from working on python2.2/2.3? Index: Makefile =================================================================== RCS file: /cvsroot/buildbot/buildbot/Makefile,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- Makefile 3 Jan 2006 09:26:41 -0000 1.18 +++ Makefile 16 Jul 2006 03:34:21 -0000 1.19 @@ -33,8 +33,11 @@ .PHONY: docs apidocs paper docs: $(MAKE) -C docs buildbot.info + apidocs: PYTHONPATH=.:$(T) python docs/epyrun -o docs/reference +some-apidocs: + PYTHONPATH=.:$(T) python docs/epyrun -o docs/reference --modules $(EPYDOCS) paper: $(MAKE) -C docs/PyCon-2003 all From warner at users.sourceforge.net Fri Jul 21 07:40:26 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Jul 2006 07:40:26 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.687,1.688 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv12539 Modified Files: ChangeLog Log Message: [project @ docs: add a table of classes that are useful in master.cfg] Original author: warner at lothar.com Date: 2006-07-21 05:31:34 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.687 retrieving revision 1.688 diff -u -d -r1.687 -r1.688 --- ChangeLog 16 Jul 2006 03:34:21 -0000 1.687 +++ ChangeLog 21 Jul 2006 07:40:24 -0000 1.688 @@ -1,3 +1,8 @@ +2006-07-20 Brian Warner + + * docs/buildbot.texinfo (Index of Useful Classes): add a table of + classes that are useful in master.cfg + 2006-07-15 Brian Warner * Makefile (some-apidocs): new target to build only some epydocs From warner at users.sourceforge.net Fri Jul 21 07:40:26 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Jul 2006 07:40:26 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.58,1.59 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv12539/docs Modified Files: buildbot.texinfo Log Message: [project @ docs: add a table of classes that are useful in master.cfg] Original author: warner at lothar.com Date: 2006-07-21 05:31:34 Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.58 retrieving revision 1.59 diff -u -d -r1.58 -r1.59 --- buildbot.texinfo 16 Jul 2006 00:59:05 -0000 1.58 +++ buildbot.texinfo 21 Jul 2006 07:40:24 -0000 1.59 @@ -44,6 +44,7 @@ * Command-line tool:: * Resources:: Getting help. * Developer's Appendix:: +* Index of Useful Classes:: * Index:: Complete index. @detailmenu @@ -5066,7 +5067,7 @@ archives are available at @uref{http://lists.sourceforge.net/lists/listinfo/buildbot-devel} - at node Developer's Appendix, Index, Resources, Top + at node Developer's Appendix, Index of Useful Classes, Resources, Top @unnumbered Developer's Appendix This appendix contains random notes about the implementation of the @@ -5089,7 +5090,140 @@ @code{.builders} dictionary. - at node Index, , Developer's Appendix, Top + at node Index of Useful Classes, Index, Developer's Appendix, Top + at unnumbered Index of Useful Classes + +This is a list of all user-visible classes. There are the ones that +are useful in @file{master.cfg}, the buildmaster's configuration file. +Classes that are not listed here are generally internal things that +admins are unlikely to have much use for. + + at heading Change Sources + + at table @code + at item buildbot.changes.freshcvs.FreshCVSSource + at ref{CVSToys - PBService} + + at item buildbot.changes.mail.FCMaildirSource + at ref{CVSToys - mail notification} + + at item buildbot.changes.mail.SyncmailMaildirSource + at ref{Other mail notification ChangeSources} + + at item buildbot.changes.mail.BonsaiMaildirSource + at ref{Other mail notification ChangeSources} + + at item buildbot.changes.p4poller.P4Source + at ref{P4Source} + + at item buildbot.changes.pb.PBChangeSource + at ref{PBChangeSource} + + at end table + + + at heading Schedulers and Locks + at table @code + at item buildbot.scheduler.Scheduler + at ref{Scheduler Types} + + at item buildbot.scheduler.AnyBranchScheduler + at ref{Scheduler Types} + + at item buildbot.scheduler.Dependent + at ref{Build Dependencies} + + at item buildbot.scheduler.Periodic + at ref{Scheduler Types} + + at item buildbot.scheduler.Nightly + at ref{Scheduler Types} + + at item buildbot.scheduler.Try_Jobdir + at ref{try} + at item buildbot.scheduler.Try_Userpass + at ref{try} + + at item buildbot.locks.MasterLock + at ref{Interlocks} + + at item buildbot.locks.SlaveLock + at ref{Interlocks} + + at end table + + at heading Build Factories + at table @code + at item buildbot.process.factory.BuildFactory + at ref{BuildFactory} + at item buildbot.process.factory.GNUAutoconf + at ref{GNUAutoconf} + at item buildbot.process.factory.CPAN + at ref{CPAN} + at item buildbot.process.factory.Distutils + at ref{Python distutils} + at item buildbot.process.factory.Trial + at ref{Python/Twisted/trial projects} + at item buildbot.process.factory.BasicBuildFactory + at item buildbot.process.factory.QuickBuildFactory + at item buildbot.process.factory.BasicSVN + at end table + + at heading Build Steps + at table @code + at item buildbot.process.step.ShellCommand + at ref{ShellCommand} + at item buildbot.process.step.TreeSize + at item buildbot.process.step.CVS + at c TODO: this CVS link is broken, it goes to a CVSToys page for some reason + at ref{CVS} + at item buildbot.process.step.SVN + at ref{SVN} + at item buildbot.process.step.Darcs + at ref{Darcs} + at item buildbot.process.step.Git + at item buildbot.process.step.Arch + at ref{Arch} + at item buildbot.process.step.Bazaar + at ref{Bazaar} + at item buildbot.process.step.Mercurial + at ref{Mercurial} + at item buildbot.process.step.P4 + at c TODO: this P4 like goes to P4Source by mistake + at ref{P4} + at item buildbot.process.step.P4Sync + at item buildbot.process.step.Configure + at ref{Configure} + at item buildbot.process.step.Compile + at ref{Compile} + at item buildbot.process.step.Test + at ref{Test} + at item buildbot.process.step_twisted.HLint + at item buildbot.process.step_twisted.Trial + at item buildbot.process.step_twisted.ProcessDocs + at item buildbot.process.step_twisted.BuildDebs + at item buildbot.process.step_twisted.RemovePYCs + at item buildbot.process.maxq.MaxQ + at end table + + + at heading Status Targets + at table @code + + at item buildbot.status.html.Waterfall + at ref{HTML Waterfall} + + at item buildbot.status.mail.MailNotifier + + at item buildbot.status.words.IRC + at ref{IRC Bot} + + at item buildbot.status.client.PBListener + at ref{PBListener} + + at end table + + at node Index, , Index of Useful Classes, Top @unnumbered Index @printindex cp From warner at users.sourceforge.net Fri Jul 21 07:40:44 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Jul 2006 07:40:44 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.688,1.689 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv12595 Modified Files: ChangeLog Log Message: [project @ sample.cfg: simplify the sample BuildFactory] Original author: warner at lothar.com Date: 2006-07-21 07:38:32 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.688 retrieving revision 1.689 diff -u -d -r1.688 -r1.689 --- ChangeLog 21 Jul 2006 07:40:24 -0000 1.688 +++ ChangeLog 21 Jul 2006 07:40:42 -0000 1.689 @@ -1,5 +1,8 @@ 2006-07-20 Brian Warner + * buildbot/scripts/sample.cfg: simplify the sample BuildFactory, + which runs the buildbot unit tests + * docs/buildbot.texinfo (Index of Useful Classes): add a table of classes that are useful in master.cfg From warner at users.sourceforge.net Fri Jul 21 07:40:44 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Jul 2006 07:40:44 +0000 Subject: [Buildbot-commits] buildbot/buildbot/scripts sample.cfg,1.10,1.11 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/scripts In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv12595/buildbot/scripts Modified Files: sample.cfg Log Message: [project @ sample.cfg: simplify the sample BuildFactory] Original author: warner at lothar.com Date: 2006-07-21 07:38:32 Index: sample.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/sample.cfg,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- sample.cfg 16 Jul 2006 03:34:05 -0000 1.10 +++ sample.cfg 21 Jul 2006 07:40:42 -0000 1.11 @@ -94,13 +94,13 @@ builders = [] from buildbot.process import step, factory -s = factory.s -source = s(step.CVS, cvsroot=cvsroot, cvsmodule=cvsmodule, login="", +f1 = factory.BuildFactory() +f1.addStep(step.CVS, + cvsroot=cvsroot, cvsmodule=cvsmodule, login="", mode="copy") -f1 = factory.Trial(source, tests="buildbot.test", - # enable this if you've installed buildbot-test-vc-1.tar.gz - #env={'BUILDBOT_TEST_VC': "~/incoming"}, - ) +f1.addStep(step.Compile, command=["./setup.py", "build"]) +f1.addStep(step_twisted.Trial, testpath=".") + b1 = {'name': "buildbot-full", 'slavename': "bot1name", 'builddir': "full", @@ -124,9 +124,13 @@ # c['status'].append(mail.MailNotifier(fromaddr="buildbot at localhost", # extraRecipients=["builds at example.com"], # sendToInterestedUsers=False)) +# # from buildbot.status import words # c['status'].append(words.IRC(host="irc.example.com", nick="bb", # channels=["#example"])) +# +# from buildbot.status import client +# c['status'].append(client.PBListener(9988)) ####### DEBUGGING OPTIONS From warner at users.sourceforge.net Mon Jul 24 22:09:22 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Jul 2006 22:09:22 +0000 Subject: [Buildbot-commits] buildbot/buildbot master.py,1.93,1.94 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv19556/buildbot Modified Files: master.py Log Message: [project @ check for duplicate Scheduler names when loading the config file] Original author: warner at lothar.com Date: 2006-07-24 22:01:13 Index: master.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/master.py,v retrieving revision 1.93 retrieving revision 1.94 diff -u -d -r1.93 -r1.94 --- master.py 16 Jul 2006 00:59:05 -0000 1.93 +++ master.py 24 Jul 2006 22:09:20 -0000 1.94 @@ -759,10 +759,20 @@ % (b['name'], b['builddir'])) dirnames.append(b['builddir']) + schedulernames = [] for s in schedulers: for b in s.listBuilderNames(): assert b in buildernames, \ "%s uses unknown builder %s" % (s, b) + if s.name in schedulernames: + # TODO: schedulers share a namespace with other Service + # children of the BuildMaster node, like status plugins, the + # Manhole, the ChangeMaster, and the BotMaster (although most + # of these don't have names) + msg = ("Schedulers must have unique names, but " + "'%s' was a duplicate" + s.name) + raise ValueError(msg) + schedulernames.append(s.name) # assert that all locks used by the Builds and their Steps are # uniquely named. From warner at users.sourceforge.net Mon Jul 24 22:09:22 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Jul 2006 22:09:22 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.689,1.690 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv19556 Modified Files: ChangeLog Log Message: [project @ check for duplicate Scheduler names when loading the config file] Original author: warner at lothar.com Date: 2006-07-24 22:01:13 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.689 retrieving revision 1.690 diff -u -d -r1.689 -r1.690 --- ChangeLog 21 Jul 2006 07:40:42 -0000 1.689 +++ ChangeLog 24 Jul 2006 22:09:19 -0000 1.690 @@ -1,3 +1,10 @@ +2006-07-24 Brian Warner + + * buildbot/master.py (BuildMaster.loadConfig): check for duplicate + Scheduler names, since they cause setServiceParent to explode + later. + * buildbot/test/test_config.py (ConfigTest._testSchedulers_7): test it + 2006-07-20 Brian Warner * buildbot/scripts/sample.cfg: simplify the sample BuildFactory, From warner at users.sourceforge.net Mon Jul 24 22:09:22 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Jul 2006 22:09:22 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_config.py, 1.35, 1.36 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv19556/buildbot/test Modified Files: test_config.py Log Message: [project @ check for duplicate Scheduler names when loading the config file] Original author: warner at lothar.com Date: 2006-07-24 22:01:13 Index: test_config.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_config.py,v retrieving revision 1.35 retrieving revision 1.36 diff -u -d -r1.35 -r1.36 --- test_config.py 7 Jun 2006 16:53:19 -0000 1.35 +++ test_config.py 24 Jul 2006 22:09:20 -0000 1.36 @@ -622,9 +622,9 @@ # reloading the same config file should leave the schedulers in place d = self.buildmaster.loadConfig(newcfg) - d.addCallback(self._testschedulers_6, sch) + d.addCallback(self._testSchedulers_6, sch) return d - def _testschedulers_6(self, res, sch1): + def _testSchedulers_6(self, res, sch1): sch2 = self.buildmaster.allSchedulers() self.failUnlessEqual(len(sch2), 2) sch1.sort() @@ -634,6 +634,17 @@ self.failUnlessIdentical(sch1[1], sch2[1]) self.failUnlessIdentical(sch1[0].parent, self.buildmaster) self.failUnlessIdentical(sch1[1].parent, self.buildmaster) + badcfg = self.schedulersCfg + \ +""" +c['schedulers'] = [Scheduler('dup', None, 60, []), + Scheduler('dup', None, 60, [])] +""" + d = defer.maybeDeferred(self.buildmaster.loadConfig, badcfg) + d.addBoth(self._testSchedulers_7) + return d + def _testSchedulers_7(self, res): + self.shouldBeFailure(res, ValueError) + def testBuilders(self): From warner at users.sourceforge.net Mon Jul 31 08:31:35 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 31 Jul 2006 08:31:35 +0000 Subject: [Buildbot-commits] buildbot .darcs-boring, 1.1, 1.2 ChangeLog, 1.690, 1.691 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv6265 Modified Files: .darcs-boring ChangeLog Log Message: [project @ docs: add lots of diagrams describing system architecture] Original author: warner at lothar.com Date: 2006-07-31 08:29:33 Index: .darcs-boring =================================================================== RCS file: /cvsroot/buildbot/buildbot/.darcs-boring,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- .darcs-boring 28 May 2006 07:56:02 -0000 1.1 +++ .darcs-boring 31 Jul 2006 08:31:32 -0000 1.2 @@ -39,8 +39,11 @@ ^_trial_temp($|/) # tree-local stuff -^docs/buildbot.html$ -^docs/buildbot.info$ +^docs/buildbot\.html$ +^docs/buildbot\.info$ +^docs/buildbot\.ps$ +^docs/images/.*\.png$ +^docs/images/.*\.eps$ ^dist($|/) ^debian($|/) Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.690 retrieving revision 1.691 diff -u -d -r1.690 -r1.691 --- ChangeLog 24 Jul 2006 22:09:19 -0000 1.690 +++ ChangeLog 31 Jul 2006 08:31:33 -0000 1.691 @@ -1,3 +1,15 @@ +2006-07-31 Brian Warner + + * docs/buildbot.texinfo (System Architecture): Finally add lots of + diagrams to describe how the whole system fits together. The + images themselves are kept in SVG files, with ascii-art versions + in corresponding .txt files. Texinfo knows how to interpolate the + text version into .info files, reference the .png versions from + .html files, and include .eps versions in the .ps format. + * docs/images/Makefile: tools to create .png and .eps + * docs/images/*.svg: created pictures with Inkscape. + * .darcs-boring: ignore the generated .eps and .png files + 2006-07-24 Brian Warner * buildbot/master.py (BuildMaster.loadConfig): check for duplicate From warner at users.sourceforge.net Mon Jul 31 08:31:35 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 31 Jul 2006 08:31:35 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.59,1.60 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv6265/docs Modified Files: buildbot.texinfo Log Message: [project @ docs: add lots of diagrams describing system architecture] Original author: warner at lothar.com Date: 2006-07-31 08:29:33 Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.59 retrieving revision 1.60 diff -u -d -r1.59 -r1.60 --- buildbot.texinfo 21 Jul 2006 07:40:24 -0000 1.59 +++ buildbot.texinfo 31 Jul 2006 08:31:33 -0000 1.60 @@ -56,6 +56,12 @@ * System Architecture:: * Control Flow:: +System Architecture + +* BuildSlave Connections:: +* Buildmaster Architecture:: +* Status Delivery Architecture:: + Installation * Requirements:: @@ -320,12 +326,12 @@ The Buildbot consists of a single @code{buildmaster} and one or more @code{buildslaves}, connected in a star topology. The buildmaster -makes all decisions about what and when to build. It sends commands to -be run on the build slaves, which simply execute the commands and -return the results. (certain steps involve more local decision making, -where the overhead of sending a lot of commands back and forth would -be inappropriate, but in general the buildmaster is responsible for -everything). +makes all decisions about what, when, and how to build. It sends +commands to be run on the build slaves, which simply execute the +commands and return the results. (certain steps involve more local +decision making, where the overhead of sending a lot of commands back +and forth would be inappropriate, but in general the buildmaster is +responsible for everything). The buildmaster is usually fed @code{Changes} by some sort of version control system @xref{Change Sources}, which may cause builds to be @@ -333,24 +339,139 @@ produced, which are then sent to any registered Status Targets @xref{Status Delivery}. - at ifinfo - at smallexample - at group - TODO: picture of change sources, master, slaves, status targets - should look like docs/PyCon-2003/sources/overview.svg - at end group - at end smallexample - at end ifinfo - at ifnotinfo - at c @image{images/overview} - at end ifnotinfo + at c @image{FILENAME, WIDTH, HEIGHT, ALTTEXT, EXTENSION} + at image{images/overview,,,Overview Diagram,} The buildmaster is configured and maintained by the ``buildmaster admin'', who is generally the project team member responsible for build process issues. Each buildslave is maintained by a ``buildslave admin'', who do not need to be quite as involved. Generally slaves are run by anyone who has an interest in seeing the project work well on -their platform. +their favorite platform. + + at menu +* BuildSlave Connections:: +* Buildmaster Architecture:: +* Status Delivery Architecture:: + at end menu + + at node BuildSlave Connections, Buildmaster Architecture, System Architecture, System Architecture + at subsection BuildSlave Connections + +The buildslaves are typically run on a variety of separate machines, +at least one per platform of interest. These machines connect to the +buildmaster over a TCP connection to a publically-visible port. As a +result, the buildslaves can live behind a NAT box or similar +firewalls, as long as they can get to buildmaster. The TCP connections +are initiated by the buildslave and accepted by the buildmaster, but +commands and results travel both ways within this connection. The +buildmaster is always in charge, so all commands travel exclusively +from the buildmaster to the buildslave. + +To perform builds, the buildslaves must typically obtain source code +from a CVS/SVN/etc repository. Therefore they must also be able to +reach the repository. The buildmaster provides instructions for +performing builds, but does not provide the source code itself. + + at image{images/slaves,,,BuildSlave Connections,} + + at node Buildmaster Architecture, Status Delivery Architecture, BuildSlave Connections, System Architecture + at subsection Buildmaster Architecture + +The Buildmaster consists of several pieces: + + at image{images/master,,,BuildMaster Architecture,} + + at itemize @bullet + + at item +Change Sources, which create a Change object each time something is +modified in the VC repository. Most ChangeSources listen for messages +from a hook script of some sort. Some sources actively poll the +repository on a regular basis. All Changes are fed to the Schedulers. + + at item +Schedulers, which decide when builds should be performed. They collect +Changes into BuildRequests, which are then queued for delivery to +Builders until a buildslave is available. + + at item +Builders, which control exactly @emph{how} each build is performed +(with a series of BuildSteps, configured in a BuildFactory). Each +Build is run on a single buildslave. + + at item +Status plugins, which deliver information about the build results +through protocols like HTTP, mail, and IRC. + + at end itemize + + at image{images/slavebuilder,,,SlaveBuilders,} + +Each Builder is configured with a list of BuildSlaves that it will use +for its builds. These buildslaves are expected to behave identically: +the only reason to use multiple BuildSlaves for a single Builder is to +provide a measure of load-balancing. + +Within a single BuildSlave, each Builder creates its own SlaveBuilder +instance. These SlaveBuilders operate independently from each other. +Each gets its own base directory to work in. It is quite common to +have many Builders sharing the same buildslave. For example, there +might be two buildslaves: one for i386, and a second for PowerPC. +There may then be a pair of Builders that do a full compile/test run, +one for each architecture, and a lone Builder that creates snapshot +source tarballs if the full builders complete successfully. The full +builders would each run on a single buildslave, whereas the tarball +creation step might run on either buildslave (since the platform +doesn't matter when creating source tarballs). In this case, the +mapping would look like: + + at example +Builder(full-i386) -> BuildSlaves(slave-i386) +Builder(full-ppc) -> BuildSlaves(slave-ppc) +Builder(source-tarball) -> BuildSlaves(slave-i386, slave-ppc) + at end example + +and each BuildSlave would have two SlaveBuilders inside it, one for a +full builder, and a second for the source-tarball builder. + +Once a SlaveBuilder is available, the Builder pulls one or more +BuildRequests off its incoming queue. (It may pull more than one if it +determines that it can merge the requests together; for example, there +may be multiple requests to build the current HEAD revision). These +requests are merged into a single Build instance, which includes the +SourceStamp that describes what exact version of the source code +should be used for the build. The Build is then assigned to a +SlaveBuilder and the build begins. + + + at node Status Delivery Architecture, , Buildmaster Architecture, System Architecture + at subsection Status Delivery Architecture + +The buildmaster maintains a central Status object, to which various +status plugins are connected. Through this Status object, a full +hierarchy of build status objects can be obtained. + + at image{images/status,,,Status Delivery,} + +The configuration file controls which status plugins are active. Each +status plugin gets a reference to the top-level Status object. From +there they can request information on each Builder, Build, Step, and +LogFile. This query-on-demand interface is used by the html.Waterfall +plugin to create the main status page each time a web browser hits the +main URL. + +The status plugins can also subscribe to hear about new Builds as they +occur: this is used by the MailNotifier to create new email messages +for each recently-completed Build. + +The Status object records the status of old builds on disk in the +buildmaster's base directory. This allows it to return information +about historical builds. + +There are also status objects that correspond to Schedulers and +BuildSlaves. These allow status plugins to report information about +upcoming builds, and the online/offline status of each buildslave. @node Control Flow, , System Architecture, Introduction From warner at users.sourceforge.net Mon Jul 31 08:31:35 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 31 Jul 2006 08:31:35 +0000 Subject: [Buildbot-commits] buildbot/docs/images Makefile, NONE, 1.1 master.svg, NONE, 1.1 master.txt, NONE, 1.1 overview.svg, NONE, 1.1 overview.txt, NONE, 1.1 slavebuilder.svg, NONE, 1.1 slavebuilder.txt, NONE, 1.1 slaves.svg, NONE, 1.1 slaves.txt, NONE, 1.1 status.svg, NONE, 1.1 status.txt, NONE, 1.1 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/images In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv6265/docs/images Added Files: Makefile master.svg master.txt overview.svg overview.txt slavebuilder.svg slavebuilder.txt slaves.svg slaves.txt status.svg status.txt Log Message: [project @ docs: add lots of diagrams describing system architecture] Original author: warner at lothar.com Date: 2006-07-31 08:29:33 --- NEW FILE: Makefile --- all: overview.png overview.eps slaves.png slaves.eps slavebuilder.png slavebuilder.eps master.png master.eps status.png status.eps %.png: %.svg inkscape -b white --export-png $@ $< %.eps: %.svg inkscape --export-eps $@ $< clean: rm -f *.png *.eps --- NEW FILE: master.svg --- image/svg+xml ChangeSource Scheduler Builder Builder Builder buildrequest buildrequest buildrequest Change Change --- NEW FILE: master.txt --- +---------------+ | Change Source |----->----+ +---------------+ | Changes | +---------------+ v | Change Source |----->----+ +---------------+ v +-----+-------+ | | v v +-----------+ +-----------+ | Scheduler | | Scheduler | +-----------+ +-----------+ | | | +------+---------+ +---+ +-----+ | | | | v | | Build : : : v v : Request : : : : | : ---- : : : | : ---- : : ---- : | +======+ +======+ : v : | | : : v v : : +---------+ +---------+ :queue : | Builder | | Builder | +======+ +---------+ +---------+ | v +---------+ | Builder | +---------+ --- NEW FILE: overview.svg --- image/svg+xml BuildMaster Build Slave Commands CVS Changes Browser Status Client IRC email Build Status Results SVN Darcs .. etc --- NEW FILE: overview.txt --- +------------------+ +-----------+ | |---------->| Browser | | BuildMaster | +-----------+ Changes | |--------------->+--------+ +----------->| | Build Status | email | | | |------------+ +--------+ | | |-------+ | +---------------+ | +------------------+ | +---->| Status Client | +----------+ | ^ | ^ | +---------------+ | Change | | | C| | | +-----+ | Sources | | | o| | +------------>| IRC | | | | | m| |R +-----+ | CVS | v | m| |e | SVN | +---------+ a| |s | Darcs | | Build | n| |u | .. etc | | Slave | d| |l | | +---------+ s| |t | | v |s +----------+ +---------+ | Build | | Slave | +---------+ --- NEW FILE: slavebuilder.svg --- image/svg+xml Builder Builder buildrequest buildrequest build BuildSlave BuildSlave SlaveBuilder SlaveBuilder SlaveBuilder SlaveBuilder build build build Builder build --- NEW FILE: slavebuilder.txt --- +-----------------+ | BuildSlave | | | | | +-------+ | +------------+ | |Builder|----Build----->|SlaveBuilder| | +-------+ | +------------+ | | | | +------------+ | +-Build---->|SlaveBuilder| | | | +------------+ | +-------+ | | | |Builder|---+ +-----------------+ +-------+ | | | +-----------------+ Build | BuildSlave | | | | | | | | | +------------+ | +------->|SlaveBuilder| | | +------------+ | +-------+ | | |Builder|--+ | +------------+ | +-------+ +-------->|SlaveBuilder| | | +------------+ | | | +-----------------+ --- NEW FILE: slaves.svg --- image/svg+xml BuildMaster Build Slave CVS Changes Results SVN Darcs .. etc Repository Commands NAT TCP TCP checkout/update --- NEW FILE: slaves.txt --- Repository| | BuildMaster | | (CVS/SVN)| | ^|^^^ | | | / c \ | ----------+ +------------------/--o----\-+ ^ / m ^ \ | / m | \ checkout/update --+ a | +-- | TCP| n | |TCP | | d | | | | s | | | | | | | | | | r | | | | e | -N-A-T-|- - - - -N-A-T- - - - -|- |- s-|- - - - -N-A-T- - - | | | u | | | | l | | +------------------|--|--t-|-+ | | | | s | | +----| v | | | | | | | | | | | BuildSlave | +----------------------------+ --- NEW FILE: status.svg --- image/svg+xml Builder buildrequest build Status Status Builder Status Builder Status Build Status Build Status Step Status Step File Log File Log Waterfall IRC MailNotifier WebBrowser IRCServer SMTP HTTP StatusPlugins Build Slave --- NEW FILE: status.txt --- Status Objects Status Plugins User Clients +------+ +---------+ +-----------+ |Status|<--------------+-->|Waterfall|<-------|Web Browser| +------+ | +---------+ +-----------+ | +-----+ | v v | +-------+ +-------+ | +---+ +----------+ |Builder| |Builder| +---->|IRC|<----------->IRC Server| |Status | |Status | | +---+ +----------+ +-------+ +-------+ | | +----+ | v v | +------------+ +----+ +------+ +------+ +-->|MailNotifier|---->|SMTP| |Build | |Build | +------------+ +----+ |Status| |Status| +------+ +------+ | +-----+ v v +------+ +------+ |Step | |Step | |Status| |Status| +------+ +------+ | +---+ v v +----+ +----+ |Log | |Log | |File| |File| +----+ +----+ From warner at users.sourceforge.net Mon Jul 31 08:34:50 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 31 Jul 2006 08:34:50 +0000 Subject: [Buildbot-commits] site/images - New directory Message-ID: Update of /cvsroot/buildbot/site/images In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7473/images Log Message: Directory /cvsroot/buildbot/site/images added to the repository From warner at users.sourceforge.net Mon Jul 31 08:36:54 2006 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 31 Jul 2006 08:36:54 +0000 Subject: [Buildbot-commits] site ChangeLog, 1.42, 1.43 manual-CVS.html, 1.4, 1.5 Message-ID: Update of /cvsroot/buildbot/site In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv8363 Modified Files: ChangeLog manual-CVS.html Log Message: updated manual-CVS.html, now with images Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/site/ChangeLog,v retrieving revision 1.42 retrieving revision 1.43 diff -u -d -r1.42 -r1.43 --- ChangeLog 12 Jul 2006 21:36:32 -0000 1.42 +++ ChangeLog 31 Jul 2006 08:36:52 -0000 1.43 @@ -1,3 +1,8 @@ +2006-07-31 Brian Warner + + * manual-CVS.html: updated, added images + * images/*.png: new diagrams for manual + 2006-07-12 Brian Warner * index.html: s/Ethereal/Wireshark/g, fix Browse-CVS link. Index: manual-CVS.html =================================================================== RCS file: /cvsroot/buildbot/site/manual-CVS.html,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- manual-CVS.html 25 Oct 2005 03:53:20 -0000 1.4 +++ manual-CVS.html 31 Jul 2006 08:36:52 -0000 1.5 @@ -1,15 +1,15 @@ -BuildBot Manual 0.7.0 +BuildBot Manual 0.7.3+ - +