From warner at users.sourceforge.net Sun Oct 2 23:25:12 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 02 Oct 2005 23:25:12 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.504,1.505 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30527 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-318 Creator: Brian Warner tolerate dnotify flag problems on 64-bit systems * buildbot/changes/maildir.py (Maildir.start): Tolerate OverflowError when setting up dnotify, because some 64-bit systems have problems with signed-vs-unsigned constants and trip up on the DN_MULTISHOT flag. Patch from Brad Hards. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.504 retrieving revision 1.505 diff -u -d -r1.504 -r1.505 --- ChangeLog 6 Sep 2005 22:07:07 -0000 1.504 +++ ChangeLog 2 Oct 2005 23:25:10 -0000 1.505 @@ -1,3 +1,10 @@ +2005-10-02 Brian Warner + + * buildbot/changes/maildir.py (Maildir.start): Tolerate + OverflowError when setting up dnotify, because some 64-bit systems + have problems with signed-vs-unsigned constants and trip up on the + DN_MULTISHOT flag. Patch from Brad Hards. + 2005-09-06 Fred Drake * buildbot/process/step.py (BuildStep, ShellCommand): Add From warner at users.sourceforge.net Sun Oct 2 23:25:12 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 02 Oct 2005 23:25:12 +0000 Subject: [Buildbot-commits] buildbot/buildbot/changes maildir.py,1.6,1.7 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/changes In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30527/buildbot/changes Modified Files: maildir.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-318 Creator: Brian Warner tolerate dnotify flag problems on 64-bit systems * buildbot/changes/maildir.py (Maildir.start): Tolerate OverflowError when setting up dnotify, because some 64-bit systems have problems with signed-vs-unsigned constants and trip up on the DN_MULTISHOT flag. Patch from Brad Hards. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: maildir.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/changes/maildir.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- maildir.py 10 Aug 2005 07:06:11 -0000 1.6 +++ maildir.py 2 Oct 2005 23:25:10 -0000 1.7 @@ -52,8 +52,10 @@ try: self.dnotify = DNotify(self.newdir, self.dnotify_callback, [DNotify.DN_CREATE]) - except IOError: - # probably linux<2.4.19, which doesn't support dnotify + except (IOError, OverflowError): + # IOError is probably linux<2.4.19, which doesn't support + # dnotify. OverflowError will occur on some 64-bit machines + # because of a python bug print "DNotify failed, falling back to polling" have_dnotify = 0 From warner at users.sourceforge.net Fri Oct 7 18:37:23 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Oct 2005 18:37:23 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process base.py,1.58,1.59 builder.py,1.30,1.31 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5665/buildbot/process Modified Files: base.py builder.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-320 Creator: Brian Warner remove inappropriate already-attached warning * buildbot/process/builder.py (Builder.attached): remove the already-attached warning, this situation is normal. Add some comments explaining it. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: base.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/base.py,v retrieving revision 1.58 retrieving revision 1.59 diff -u -d -r1.58 -r1.59 --- base.py 17 Aug 2005 02:15:37 -0000 1.58 +++ base.py 7 Oct 2005 18:37:21 -0000 1.59 @@ -258,6 +258,11 @@ first Step. It returns a Deferred which will fire when the build finishes. This Deferred is guaranteed to never errback.""" + # we are taking responsibility for watching the connection to the + # remote. This responsibility was held by the Builder until our + # startBuild was called, and will not return to them until we fire + # the Deferred returned by this method. + log.msg("%s.startBuild" % self) self.build_status = build_status self.slavebuilder = slavebuilder Index: builder.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/builder.py,v retrieving revision 1.30 retrieving revision 1.31 diff -u -d -r1.30 -r1.31 --- builder.py 31 Aug 2005 01:51:43 -0000 1.30 +++ builder.py 7 Oct 2005 18:37:21 -0000 1.31 @@ -307,11 +307,24 @@ """ for s in self.slaves.keys(): if s.slave == slave: - # already attached to them. TODO: how does this ever get - # reached? - log.msg("%s.attached: WEIRD slave %s already attached" - % (self, slave)) + # already attached to them. This is fairly common, since + # attached() gets called each time we receive the builder + # list from the slave, and we ask for it each time we add or + # remove a builder. So if the slave is hosting builders + # A,B,C, and the config file changes A, we'll remove A and + # re-add it, triggering two builder-list requests, getting + # two redundant calls to attached() for B, and another two + # for C. + # + # Therefore, when we see that we're already attached, we can + # just ignore it. TODO: build a diagram of the state + # transitions here, I'm concerned about sb.attached() failing + # and leaving self.slaves[sb] stuck at 'attaching', and about + # the detached() message arriving while there's some + # transition pending such that the response to the transition + # re-vivifies self.slaves[sb] return defer.succeed(self) + sb = SlaveBuilder(self) self.slaves[sb] = "attaching" d = sb.attached(slave, remote, commands) @@ -331,7 +344,8 @@ def _not_attached(self, why, slave): # already log.err'ed by SlaveBuilder._attachFailure # TODO: make this .addSlaveEvent? - # TODO: remove from self.slaves + # TODO: remove from self.slaves (except that detached() should get + # run first, right?) self.builder_status.addPointEvent(['failed', 'connect', slave.slave.slavename]) # TODO: add an HTMLLogFile of the exception @@ -467,6 +481,12 @@ sb.finishBuild() if sb in self.slaves: self.slaves[sb] = "idle" + else: + # if the startBuild message failed because we lost the slave, our + # detacted() method will have already fired, removing the + # SlaveBuilder from self.slaves . This test is here to make sure + # we don't re-create the old self.slaves[sb] entry. + pass log.msg("re-queueing the BuildRequest") self.building.remove(build) From warner at users.sourceforge.net Fri Oct 7 18:37:23 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Oct 2005 18:37:23 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.505,1.506 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5665 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-320 Creator: Brian Warner remove inappropriate already-attached warning * buildbot/process/builder.py (Builder.attached): remove the already-attached warning, this situation is normal. Add some comments explaining it. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.505 retrieving revision 1.506 diff -u -d -r1.505 -r1.506 --- ChangeLog 2 Oct 2005 23:25:10 -0000 1.505 +++ ChangeLog 7 Oct 2005 18:37:21 -0000 1.506 @@ -1,3 +1,9 @@ +2005-10-07 Brian Warner + + * buildbot/process/builder.py (Builder.attached): remove the + already-attached warning, this situation is normal. Add some + comments explaining it. + 2005-10-02 Brian Warner * buildbot/changes/maildir.py (Maildir.start): Tolerate From warner at users.sourceforge.net Fri Oct 7 18:45:44 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Oct 2005 18:45:44 +0000 Subject: [Buildbot-commits] buildbot/buildbot/slave bot.py,1.14,1.15 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/slave In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7377/buildbot/slave Modified Files: bot.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-322 Creator: Brian Warner inherit from StatusReceiver properly * buildbot/slave/bot.py (SlaveBuilder.activity): survive bot.SlaveBuilder being disowned in the middle of a build * buildbot/status/base.py (StatusReceiverMultiService): oops, make this inherit from StatusReceiver. Also upcall in __init__. This fixes the embarrasing crash when the new buildSetSubmitted method is invoked and Waterfall/etc don't implement their own. * buildbot/test/test_run.py: add a TODO note about a test to catch just this sort of thing. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: bot.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/bot.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- bot.py 19 Jul 2005 23:12:00 -0000 1.14 +++ bot.py 7 Oct 2005 18:45:41 -0000 1.15 @@ -72,6 +72,10 @@ def setServiceParent(self, parent): service.Service.setServiceParent(self, parent) self.bot = self.parent + # note that self.parent will go away when the buildmaster's config + # file changes and this Builder is removed (possibly because it has + # been changed, so the Builder will be re-added again in a moment). + # This may occur during a build, while a step is running. def setBuilddir(self, builddir): assert self.parent @@ -86,10 +90,12 @@ self.stopCommand() def activity(self): - #bot = self.parent - #buildslave = bot.parent - #bf = buildslave.bf - self.parent.parent.bf.activity() + bot = self.parent + if bot: + buildslave = bot.parent + if buildslave: + bf = buildslave.bf + bf.activity() def remote_setMaster(self, remote): self.remote = remote From warner at users.sourceforge.net Fri Oct 7 18:45:44 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Oct 2005 18:45:44 +0000 Subject: [Buildbot-commits] buildbot/buildbot master.py,1.80,1.81 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7377/buildbot Modified Files: master.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-322 Creator: Brian Warner inherit from StatusReceiver properly * buildbot/slave/bot.py (SlaveBuilder.activity): survive bot.SlaveBuilder being disowned in the middle of a build * buildbot/status/base.py (StatusReceiverMultiService): oops, make this inherit from StatusReceiver. Also upcall in __init__. This fixes the embarrasing crash when the new buildSetSubmitted method is invoked and Waterfall/etc don't implement their own. * buildbot/test/test_run.py: add a TODO note about a test to catch just this sort of thing. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: master.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/master.py,v retrieving revision 1.80 retrieving revision 1.81 diff -u -d -r1.80 -r1.81 --- master.py 17 Aug 2005 02:15:36 -0000 1.80 +++ master.py 7 Oct 2005 18:45:42 -0000 1.81 @@ -921,6 +921,8 @@ "\n".join(diffs))) # TODO: if the basedir was changed, we probably need to # make a new statusbag + # TODO: if a slave is connected and we're re-using the + # same slave, try to avoid a disconnect/reconnect cycle. statusbag = old.builder_status statusbag.saveYourself() # seems like a good idea d = self.botmaster.removeBuilder(old) From warner at users.sourceforge.net Fri Oct 7 18:45:44 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Oct 2005 18:45:44 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_run.py,1.33,1.34 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7377/buildbot/test Modified Files: test_run.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-322 Creator: Brian Warner inherit from StatusReceiver properly * buildbot/slave/bot.py (SlaveBuilder.activity): survive bot.SlaveBuilder being disowned in the middle of a build * buildbot/status/base.py (StatusReceiverMultiService): oops, make this inherit from StatusReceiver. Also upcall in __init__. This fixes the embarrasing crash when the new buildSetSubmitted method is invoked and Waterfall/etc don't implement their own. * buildbot/test/test_run.py: add a TODO note about a test to catch just this sort of thing. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: test_run.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_run.py,v retrieving revision 1.33 retrieving revision 1.34 diff -u -d -r1.33 -r1.34 --- test_run.py 19 Jul 2005 23:11:58 -0000 1.33 +++ test_run.py 7 Oct 2005 18:45:42 -0000 1.34 @@ -731,3 +731,9 @@ # add a new builder, which causes the basedir list to be reloaded d = self.master.loadConfig(config_4_newbuilder) return d + +# TODO: test everything, from Change submission to Scheduler to Build to +# Status. Use all the status types. Specifically I want to catch recurrences +# of the bug where I forgot to make Waterfall inherit from StatusReceiver +# such that buildSetSubmitted failed. + From warner at users.sourceforge.net Fri Oct 7 18:45:45 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Oct 2005 18:45:45 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.506,1.507 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7377 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-322 Creator: Brian Warner inherit from StatusReceiver properly * buildbot/slave/bot.py (SlaveBuilder.activity): survive bot.SlaveBuilder being disowned in the middle of a build * buildbot/status/base.py (StatusReceiverMultiService): oops, make this inherit from StatusReceiver. Also upcall in __init__. This fixes the embarrasing crash when the new buildSetSubmitted method is invoked and Waterfall/etc don't implement their own. * buildbot/test/test_run.py: add a TODO note about a test to catch just this sort of thing. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.506 retrieving revision 1.507 diff -u -d -r1.506 -r1.507 --- ChangeLog 7 Oct 2005 18:37:21 -0000 1.506 +++ ChangeLog 7 Oct 2005 18:45:42 -0000 1.507 @@ -1,5 +1,15 @@ 2005-10-07 Brian Warner + * buildbot/slave/bot.py (SlaveBuilder.activity): survive + bot.SlaveBuilder being disowned in the middle of a build + + * buildbot/status/base.py (StatusReceiverMultiService): oops, make + this inherit from StatusReceiver. Also upcall in __init__. This + fixes the embarrasing crash when the new buildSetSubmitted method + is invoked and Waterfall/etc don't implement their own. + * buildbot/test/test_run.py: add a TODO note about a test to catch + just this sort of thing. + * buildbot/process/builder.py (Builder.attached): remove the already-attached warning, this situation is normal. Add some comments explaining it. From warner at users.sourceforge.net Fri Oct 7 18:45:45 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 07 Oct 2005 18:45:45 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status base.py,1.3,1.4 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7377/buildbot/status Modified Files: base.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-322 Creator: Brian Warner inherit from StatusReceiver properly * buildbot/slave/bot.py (SlaveBuilder.activity): survive bot.SlaveBuilder being disowned in the middle of a build * buildbot/status/base.py (StatusReceiverMultiService): oops, make this inherit from StatusReceiver. Also upcall in __init__. This fixes the embarrasing crash when the new buildSetSubmitted method is invoked and Waterfall/etc don't implement their own. * buildbot/test/test_run.py: add a TODO note about a test to catch just this sort of thing. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: base.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/base.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- base.py 31 Aug 2005 08:04:36 -0000 1.3 +++ base.py 7 Oct 2005 18:45:42 -0000 1.4 @@ -58,12 +58,16 @@ def builderRemoved(self, builderName): pass -class StatusReceiverMultiService(service.MultiService, util.ComparableMixin): +class StatusReceiverMultiService(StatusReceiver, service.MultiService, + util.ComparableMixin): if implements: implements(IStatusReceiver) else: __implements__ = IStatusReceiver, service.MultiService.__implements__ + def __init__(self): + service.MultiService.__init__(self) + class StatusReceiverPerspective(StatusReceiver, pbutil.NewCredPerspective): if implements: From warner at users.sourceforge.net Fri Oct 14 19:32:58 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:32:58 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.66,1.67 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30432/buildbot/status Modified Files: builder.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-324 Creator: Brian Warner BuildSet did not report failure correctly, breaking Dependent builds * buildbot/buildset.py (BuildSet): fix bug where BuildSet did not report failure correctly, causing Dependent builds to run when they shouldn't have. * buildbot/status/builder.py (BuildSetStatus): same * buildbot/test/test_buildreq.py (Set.testBuildSet): verify it (Set.testSuccess): test the both-pass case too * buildbot/test/test_dependencies.py (Dependencies.testRun_Fail): fix this test: it was ending too early, masking the failure before (Logger): specialized StatusReceiver to make sure the dependent builds aren't even started, much less completed. Index: builder.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v retrieving revision 1.66 retrieving revision 1.67 diff -u -d -r1.66 -r1.67 --- builder.py 31 Aug 2005 02:26:34 -0000 1.66 +++ builder.py 14 Oct 2005 19:32:55 -0000 1.67 @@ -567,13 +567,17 @@ self.id = bsid self.successWatchers = [] self.finishedWatchers = [] - self.failed = False + self.stillHopeful = True self.finished = False def setBuildRequestStatuses(self, buildRequestStatuses): self.buildRequests = buildRequestStatuses def setResults(self, results): + # the build set succeeds only if all its component builds succeed self.results = results + def giveUpHope(self): + self.stillHopeful = False + def notifySuccessWatchers(self): for d in self.successWatchers: @@ -582,8 +586,6 @@ def notifyFinishedWatchers(self): self.finished = True - if not self.failed: - self.notifySuccessWatchers() for d in self.finishedWatchers: d.callback(self) self.finishedWatchers = [] @@ -607,7 +609,7 @@ return self.finished def waitUntilSuccess(self): - if self.failed or self.finished: + if self.finished or not self.stillHopeful: # the deferreds have already fired return defer.succeed(self) d = defer.Deferred() From warner at users.sourceforge.net Fri Oct 14 19:32:57 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:32:57 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.507,1.508 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30432 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-324 Creator: Brian Warner BuildSet did not report failure correctly, breaking Dependent builds * buildbot/buildset.py (BuildSet): fix bug where BuildSet did not report failure correctly, causing Dependent builds to run when they shouldn't have. * buildbot/status/builder.py (BuildSetStatus): same * buildbot/test/test_buildreq.py (Set.testBuildSet): verify it (Set.testSuccess): test the both-pass case too * buildbot/test/test_dependencies.py (Dependencies.testRun_Fail): fix this test: it was ending too early, masking the failure before (Logger): specialized StatusReceiver to make sure the dependent builds aren't even started, much less completed. Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.507 retrieving revision 1.508 diff -u -d -r1.507 -r1.508 --- ChangeLog 7 Oct 2005 18:45:42 -0000 1.507 +++ ChangeLog 14 Oct 2005 19:32:55 -0000 1.508 @@ -1,3 +1,16 @@ +2005-10-12 Brian Warner + + * buildbot/buildset.py (BuildSet): fix bug where BuildSet did not + report failure correctly, causing Dependent builds to run when + they shouldn't have. + * buildbot/status/builder.py (BuildSetStatus): same + * buildbot/test/test_buildreq.py (Set.testBuildSet): verify it + (Set.testSuccess): test the both-pass case too + * buildbot/test/test_dependencies.py (Dependencies.testRun_Fail): + fix this test: it was ending too early, masking the failure before + (Logger): specialized StatusReceiver to make sure the dependent + builds aren't even started, much less completed. + 2005-10-07 Brian Warner * buildbot/slave/bot.py (SlaveBuilder.activity): survive From warner at users.sourceforge.net Fri Oct 14 19:32:57 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:32:57 +0000 Subject: [Buildbot-commits] buildbot/buildbot buildset.py,1.2,1.3 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30432/buildbot Modified Files: buildset.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-324 Creator: Brian Warner BuildSet did not report failure correctly, breaking Dependent builds * buildbot/buildset.py (BuildSet): fix bug where BuildSet did not report failure correctly, causing Dependent builds to run when they shouldn't have. * buildbot/status/builder.py (BuildSetStatus): same * buildbot/test/test_buildreq.py (Set.testBuildSet): verify it (Set.testSuccess): test the both-pass case too * buildbot/test/test_dependencies.py (Dependencies.testRun_Fail): fix this test: it was ending too early, masking the failure before (Logger): specialized StatusReceiver to make sure the dependent builds aren't even started, much less completed. Index: buildset.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/buildset.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- buildset.py 17 Aug 2005 02:15:36 -0000 1.2 +++ buildset.py 14 Oct 2005 19:32:55 -0000 1.3 @@ -19,7 +19,7 @@ self.builderNames = builderNames self.source = source self.reason = reason - self.failed = False + self.stillHopeful = True self.status = bss = builder.BuildSetStatus(source, reason, builderNames, bsid) @@ -51,13 +51,27 @@ b.submitBuildRequest(req) def requestFinished(self, buildstatus, req): + # TODO: this is where individual build status results are aggregated + # into a BuildSet-wide status. Consider making a rule that says one + # WARNINGS results in the overall status being WARNINGS too. The + # current rule is that any FAILURE means FAILURE, otherwise you get + # SUCCESS. self.requests.remove(req) - if buildstatus.getResults() == builder.FAILURE: - if not self.failed: - self.failed = self.status.failed = True - self.status.setResults(builder.FAILURE) + results = buildstatus.getResults() + if results == builder.FAILURE: + self.status.setResults(results) + if self.stillHopeful: + # oh, cruel reality cuts deep. no joy for you. This is the + # first failure. This flunks the overall BuildSet, so we can + # notify success watchers that they aren't going to be happy. + self.stillHopeful = False + self.status.giveUpHope() self.status.notifySuccessWatchers() if not self.requests: - self.status.setResults(builder.SUCCESS) + # that was the last build, so we can notify finished watchers. If + # we haven't failed by now, we can claim success. + if self.stillHopeful: + self.status.setResults(builder.SUCCESS) + self.status.notifySuccessWatchers() self.status.notifyFinishedWatchers() From warner at users.sourceforge.net Fri Oct 14 19:32:57 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:32:57 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_dependencies.py,1.1,1.2 test_buildreq.py,1.3,1.4 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30432/buildbot/test Modified Files: test_dependencies.py test_buildreq.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-324 Creator: Brian Warner BuildSet did not report failure correctly, breaking Dependent builds * buildbot/buildset.py (BuildSet): fix bug where BuildSet did not report failure correctly, causing Dependent builds to run when they shouldn't have. * buildbot/status/builder.py (BuildSetStatus): same * buildbot/test/test_buildreq.py (Set.testBuildSet): verify it (Set.testSuccess): test the both-pass case too * buildbot/test/test_dependencies.py (Dependencies.testRun_Fail): fix this test: it was ending too early, masking the failure before (Logger): specialized StatusReceiver to make sure the dependent builds aren't even started, much less completed. Index: test_buildreq.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_buildreq.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- test_buildreq.py 17 Aug 2005 02:15:38 -0000 1.3 +++ test_buildreq.py 14 Oct 2005 19:32:55 -0000 1.4 @@ -105,6 +105,14 @@ self.failUnlessEqual(r1.reason, s.reason) self.failUnlessEqual(r1.source, s.source) + st = s.status + self.failUnlessEqual(st.getSourceStamp(), source) + self.failUnlessEqual(st.getReason(), "forced build") + self.failUnlessEqual(st.getBuilderNames(), ["a","b"]) + self.failIf(st.isFinished()) + brs = st.getBuildRequests() + self.failUnlessEqual(len(brs), 2) + res = [] d1 = s.waitUntilSuccess() d1.addCallback(lambda r: res.append(("success", r))) @@ -113,22 +121,62 @@ self.failUnlessEqual(res, []) + # the first build finishes here, with FAILURE builderstatus_a = builder.BuilderStatus("a") - builderstatus_b = builder.BuilderStatus("b") bsa = builder.BuildStatus(builderstatus_a, 1) bsa.setResults(builder.FAILURE) a.requests[0].finished(bsa) + # any FAILURE flunks the BuildSet immediately, so the + # waitUntilSuccess deferred fires right away. However, the + # waitUntilFinished deferred must wait until all builds have + # completed. self.failUnlessEqual(len(res), 1) self.failUnlessEqual(res[0][0], "success") bss = res[0][1] self.failUnless(interfaces.IBuildSetStatus(bss, None)) + self.failUnlessEqual(bss.getResults(), builder.FAILURE) + # here we finish the second build + builderstatus_b = builder.BuilderStatus("b") bsb = builder.BuildStatus(builderstatus_b, 1) bsb.setResults(builder.SUCCESS) b.requests[0].finished(bsb) + # .. which ought to fire the waitUntilFinished deferred self.failUnlessEqual(len(res), 2) self.failUnlessEqual(res[1][0], "finished") self.failUnlessEqual(res[1][1], bss) + # and finish the BuildSet overall + self.failUnless(st.isFinished()) + self.failUnlessEqual(st.getResults(), builder.FAILURE) + + def testSuccess(self): + S = buildset.BuildSet + a,b = FakeBuilder(), FakeBuilder() + # this time, both builds succeed + + source = sourcestamp.SourceStamp() + s = S(["a","b"], source, "forced build") + s.start([a,b]) + + st = s.status + self.failUnlessEqual(st.getSourceStamp(), source) + self.failUnlessEqual(st.getReason(), "forced build") + self.failUnlessEqual(st.getBuilderNames(), ["a","b"]) + self.failIf(st.isFinished()) + + builderstatus_a = builder.BuilderStatus("a") + bsa = builder.BuildStatus(builderstatus_a, 1) + bsa.setResults(builder.SUCCESS) + a.requests[0].finished(bsa) + + builderstatus_b = builder.BuilderStatus("b") + bsb = builder.BuildStatus(builderstatus_b, 1) + bsb.setResults(builder.SUCCESS) + b.requests[0].finished(bsb) + + self.failUnless(st.isFinished()) + self.failUnlessEqual(st.getResults(), builder.SUCCESS) + Index: test_dependencies.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_dependencies.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- test_dependencies.py 19 Jul 2005 23:11:58 -0000 1.1 +++ test_dependencies.py 14 Oct 2005 19:32:55 -0000 1.2 @@ -10,6 +10,7 @@ from buildbot.process.base import BuildRequest from buildbot.test.runutils import RunMixin from buildbot.twcompat import maybeWait +from buildbot.status import base config_1 = """ from buildbot import scheduler @@ -23,6 +24,12 @@ c['schedulers'] = [] c['slavePortnum'] = 0 +# upstream1 (fastfail, slowpass) +# -> downstream2 (b3, b4) +# upstream3 (slowfail, slowpass) +# -> downstream4 (b3, b4) +# -> downstream5 (b5) + s1 = scheduler.Scheduler('upstream1', None, 10, ['slowpass', 'fastfail']) s2 = scheduler.Dependent('downstream2', s1, ['b3', 'b4']) s3 = scheduler.Scheduler('upstream3', None, 10, ['fastpass', 'slowpass']) @@ -47,6 +54,16 @@ ] """ +class Logger(base.StatusReceiverMultiService): + def __init__(self, master): + base.StatusReceiverMultiService.__init__(self) + self.builds = [] + for bn in master.status.getBuilderNames(): + master.status.getBuilder(bn).subscribe(self) + + def buildStarted(self, builderName, build): + self.builds.append(builderName) + class Dependencies(RunMixin, unittest.TestCase): def setUp(self): RunMixin.setUp(self) @@ -67,6 +84,10 @@ # that's it, just make sure this config file is loaded successfully def testRun_Fail(self): + # add an extra status target to make pay attention to which builds + # start and which don't. + self.logger = Logger(self.master) + # kick off upstream1, which has a failing Builder and thus will not # trigger downstream3 s = self.findScheduler("upstream1") @@ -77,7 +98,7 @@ # t=2: builder 'slowpass' finishes d = defer.Deferred() d.addCallback(self._testRun_Fail_1) - reactor.callLater(3, d.callback, None) + reactor.callLater(5, d.callback, None) return maybeWait(d) def _testRun_Fail_1(self, res): @@ -94,6 +115,14 @@ self.failIf(self.status.getBuilder('b4').getLastFinishedBuild()) self.failIf(self.status.getBuilder('b5').getLastFinishedBuild()) + # in fact, none of them should have even started + self.failUnlessEqual(len(self.logger.builds), 2) + self.failUnless("slowpass" in self.logger.builds) + self.failUnless("fastfail" in self.logger.builds) + self.failIf("b3" in self.logger.builds) + self.failIf("b4" in self.logger.builds) + self.failIf("b5" in self.logger.builds) + def testRun_Pass(self): # kick off upstream3, which will fire downstream4 and then # downstream5 From warner at users.sourceforge.net Fri Oct 14 19:42:42 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:42 +0000 Subject: [Buildbot-commits] buildbot/buildbot/slave bot.py,1.15,1.16 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/slave In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/buildbot/slave Modified Files: bot.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: bot.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/bot.py,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- bot.py 7 Oct 2005 18:45:41 -0000 1.15 +++ bot.py 14 Oct 2005 19:42:39 -0000 1.16 @@ -101,7 +101,27 @@ self.remote = remote self.remote.notifyOnDisconnect(self.lostRemote) def remote_print(self, message): - log.msg("builder '%s' message from master:" % self.name, message) + log.msg("SlaveBuilder.remote_print(%s): message from master: %s" % + (self.name, message)) + if message == "ping": + return self.remote_ping() + + def remote_ping(self): + log.msg("SlaveBuilder.remote_ping(%s)" % self) + if self.bot and self.bot.parent: + debugOpts = self.bot.parent.debugOpts + if debugOpts.get("stallPings"): + log.msg(" debug_stallPings") + timeout, timers = debugOpts["stallPings"] + d = defer.Deferred() + t = reactor.callLater(timeout, d.callback, None) + timers.append(t) + return d + if debugOpts.get("failPingOnce"): + log.msg(" debug_failPingOnce") + class FailPingError(pb.Error): pass + del debugOpts['failPingOnce'] + raise FailPingError("debug_failPingOnce means we should fail") def lostRemote(self, remote): log.msg("lost remote") @@ -120,7 +140,7 @@ creates a new SlaveBuild object, which holds slave-side state from one step to the next.""" self.build = SlaveBuild(self) - log.msg("startBuild") + log.msg("%s.startBuild" % self) def remote_startCommand(self, stepref, stepId, command, args): """ @@ -426,9 +446,20 @@ class BuildSlave(service.MultiService): botClass = Bot + # debugOpts is a dictionary used during unit tests. + + # debugOpts['stallPings'] can be set to a tuple of (timeout, []). Any + # calls to remote_print will stall for 'timeout' seconds before + # returning. The DelayedCalls used to implement this are stashed in the + # list so they can be cancelled later. + + # debugOpts['failPingOnce'] can be set to True to make the slaveping fail + # exactly once. + def __init__(self, host, port, name, passwd, basedir, keepalive, - usePTY, keepaliveTimeout=30): + usePTY, keepaliveTimeout=30, debugOpts={}): service.MultiService.__init__(self) + self.debugOpts = debugOpts.copy() bot = self.botClass(basedir, usePTY) bot.setServiceParent(self) self.bot = bot @@ -450,6 +481,7 @@ def stopService(self): self.bf.continueTrying = 0 + self.bf.stopTrying() service.MultiService.stopService(self) # now kill the TCP connection # twisted >2.0.1 does this for us, and leaves _connection=None From warner at users.sourceforge.net Fri Oct 14 19:42:41 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:41 +0000 Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.32,1.33 master.py,1.81,1.82 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/buildbot Modified Files: interfaces.py master.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: interfaces.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v retrieving revision 1.32 retrieving revision 1.33 diff -u -d -r1.32 -r1.33 --- interfaces.py 1 Sep 2005 20:53:21 -0000 1.32 +++ interfaces.py 14 Oct 2005 19:42:39 -0000 1.33 @@ -171,6 +171,10 @@ pass def getBuilderName(): pass + def getBuilds(): + """Return a list of IBuildStatus objects for each Build that has been + started in an attempt to satify this BuildRequest.""" + def subscribe(observer): """Register a callable that will be invoked (with a single IBuildStatus object) for each Build that is created to satisfy this @@ -211,25 +215,24 @@ def getState(): # TODO: this isn't nearly as meaningful as it used to be - """Return a tuple (state, build=None) for this Builder. 'state' is - the so-called 'big-status', indicating overall status (as opposed to + """Return a tuple (state, builds) for this Builder. 'state' is the + so-called 'big-status', indicating overall status (as opposed to which step is currently running). It is a string, one of 'offline', - 'idle', or 'building'. In the 'building' state, 'build' may be an - IBuildStatus object representing the current build (or None if the - Builder is in the pre-build ping-the-slave phase).""" + 'idle', or 'building'. 'builds' is a list of IBuildStatus objects + (possibly empty) representing the currently active builds.""" - def getSlave(): - """Return an ISlaveStatus object for the buildslave that is used by - this builder.""" + def getSlaves(): + """Return a list of ISlaveStatus objects for the buildslaves that are + used by this builder.""" def getPendingBuilds(): """Return an IBuildRequestStatus object for all upcoming builds (those which are ready to go but which are waiting for a buildslave to be available.""" - def getCurrentBuild(): - """Return an IBuildStatus object for the current build in progress. - If the state is not 'building', this will be None.""" + def getCurrentBuilds(): + """Return a list containing an IBuildStatus object for each build + currently in progress.""" # again, we could probably provide an object for 'waiting' and # 'interlocked' too, but things like the Change list might still be # subject to change @@ -360,6 +363,9 @@ # Once you know the build has finished, the following methods are legal. # Before ths build has finished, they all return None. + def getSlavename(): + """Return the name of the buildslave which handled this build.""" + def getText(): """Returns a list of strings to describe the build. These are intended to be displayed in a narrow column. If more space is @@ -807,7 +813,9 @@ def requestBuild(request): """Queue a L{buildbot.process.base.BuildRequest} object for later - building.""" + building. This returns a Deferred that fires (with an L{IBuildStatus} + instance) when the BuildRequest finishes, just as if you did + req.waitUntilFinished.""" def getPendingBuilds(): """Return a list of L{IBuildRequestControl} objects for this Builder. Index: master.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/master.py,v retrieving revision 1.81 retrieving revision 1.82 diff -u -d -r1.81 -r1.82 --- master.py 7 Oct 2005 18:45:42 -0000 1.81 +++ master.py 14 Oct 2005 19:42:39 -0000 1.82 @@ -24,7 +24,7 @@ from buildbot.twcompat import implements from buildbot.util import now from buildbot.pbutil import NewCredPerspective -from buildbot.process.builder import Builder +from buildbot.process.builder import Builder, IDLE from buildbot.status.builder import BuilderStatus, SlaveStatus, Status from buildbot.changes.changes import Change, ChangeMaster from buildbot import interfaces @@ -289,21 +289,28 @@ # which is the master-side object that defines and controls a build. # They are added by calling botmaster.addBuilder() from the startup # code. + + # self.slaves contains a ready BotPerspective instance for each + # potential buildslave, i.e. all the ones listed in the config file. + # If the slave is connected, self.slaves[slavename].slave will + # contain a RemoteReference to their Bot instance. If it is not + # connected, that attribute will hold None. self.slaves = {} # maps slavename to BotPerspective self.statusClientService = None self.watchers = {} + + # these four are convenience functions for testing + def waitUntilBuilderAttached(self, name): - # convenience function for testing b = self.builders[name] - if b.slaves: - return defer.succeed(None) + #if b.slaves: + # return defer.succeed(None) d = defer.Deferred() b.watchers['attach'].append(d) return d def waitUntilBuilderDetached(self, name): - # convenience function for testing b = self.builders.get(name) if not b or not b.slaves: return defer.succeed(None) @@ -311,11 +318,20 @@ b.watchers['detach'].append(d) return d + def waitUntilBuilderFullyDetached(self, name): + b = self.builders.get(name) + # TODO: this looks too deeply inside the Builder object + if not b or not b.slaves: + return defer.succeed(None) + d = defer.Deferred() + b.watchers['detach_all'].append(d) + return d + def waitUntilBuilderIdle(self, name): - # convenience function for testing b = self.builders[name] - for sb in b.slaves.keys(): - if b.slaves[sb] != "idle": + # TODO: this looks way too deeply inside the Builder object + for sb in b.slaves: + if sb.state != IDLE: d = defer.Deferred() b.watchers['idle'].append(d) return d @@ -349,17 +365,18 @@ if builder.name in self.builderNames: raise KeyError("muliply defined builder '%s'" % builder.name) - slavename = builder.slavename - if not self.slaves.has_key(slavename): - raise KeyError("builder %s uses undefined slave %s" % \ - (builder.name, slavename)) + for slavename in builder.slavenames: + if not self.slaves.has_key(slavename): + raise KeyError("builder %s uses undefined slave %s" % \ + (builder.name, slavename)) self.builders[builder.name] = builder self.builderNames.append(builder.name) builder.setBotmaster(self) - slave = self.slaves[slavename] - return slave.addBuilder(builder) + dl = [self.slaves[slavename].addBuilder(builder) + for slavename in builder.slavenames] + return defer.DeferredList(dl) def removeBuilder(self, builder): """Stop using a Builder. @@ -373,9 +390,10 @@ b = self.builders[builder.name] del self.builders[builder.name] self.builderNames.remove(builder.name) - slave = self.slaves.get(builder.slavename) - if slave: - return slave.removeBuilder(builder) + for slavename in builder.slavenames: + slave = self.slaves.get(slavename) + if slave: + return slave.removeBuilder(builder) return defer.succeed(None) def getPerspective(self, slavename): @@ -734,9 +752,13 @@ if type(b) is tuple: raise ValueError("builder %s must be defined with a dict, " "not a tuple" % b[0]) - if b['slavename'] not in slavenames: + if b.has_key('slavename') and b['slavename'] not in slavenames: raise ValueError("builder %s uses undefined slave %s" \ % (b['name'], b['slavename'])) + for n in b.get('slavenames', []): + if n not in slavenames: + raise ValueError("builder %s uses undefined slave %s" \ + % (b['name'], n)) if b['name'] in buildernames: raise ValueError("duplicate builder name %s" % b['name']) From warner at users.sourceforge.net Fri Oct 14 19:42:42 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:42 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status client.py,1.23,1.24 html.py,1.68,1.69 words.py,1.40,1.41 builder.py,1.67,1.68 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/buildbot/status Modified Files: client.py html.py words.py builder.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: builder.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v retrieving revision 1.67 retrieving revision 1.68 diff -u -d -r1.67 -r1.68 --- builder.py 14 Oct 2005 19:32:55 -0000 1.67 +++ builder.py 14 Oct 2005 19:42:40 -0000 1.68 @@ -645,6 +645,8 @@ return self.source def getBuilderName(self): return self.builderName + def getBuilds(self): + return self.builds def subscribe(self, observer): self.observers.append(observer) @@ -909,6 +911,7 @@ text = [] color = None results = None + slavename = None # these lists/dicts are defined here so that unserialized instances have # (empty) values. They are set in __init__ to new objects to make sure @@ -1018,6 +1021,9 @@ def getResults(self): return self.results + def getSlavename(self): + return self.slavename + def getTestResults(self): return self.testResults @@ -1094,6 +1100,9 @@ # the world about us self.builder.buildStarted(self) + def setSlavename(self, slavename): + self.slavename = slavename + def setText(self, text): assert type(text) in (list, tuple) self.text = text @@ -1280,9 +1289,7 @@ buildHorizon = 100 # forget builds beyond this stepHorizon = 50 # forget steps in builds beyond this - slavename = None category = None - currentBuild = None currentBigState = "offline" # or idle/waiting/interlocked/building nextBuildNumber = 0 basedir = None # filled in by our parent @@ -1291,27 +1298,30 @@ self.name = buildername self.category = category + self.slavenames = [] self.events = [] # these three hold Events, and are used to retrieve the current # state of the boxes. self.lastBuildStatus = None #self.currentBig = None #self.currentSmall = None + self.currentBuilds = [] self.pendingBuilds = [] self.nextBuild = None self.watchers = [] self.buildCache = [] # TODO: age builds out of the cache # persistence + # TODO: why am I not using styles.Versioned for this? def __getstate__(self): d = self.__dict__.copy() d['watchers'] = [] del d['buildCache'] - if self.currentBuild: - self.currentBuild.saveYourself() + for b in self.currentBuilds: + b.saveYourself() # TODO: push a 'hey, build was interrupted' event - del d['currentBuild'] + del d['currentBuilds'] del d['pendingBuilds'] del d['currentBigState'] del d['basedir'] @@ -1321,8 +1331,12 @@ def __setstate__(self, d): self.__dict__ = d self.buildCache = [] + self.currentBuilds = [] self.pendingBuilds = [] self.watchers = [] + if d.has_key('slavename'): + self.slavenames = [self.slavename] + del self.slavename # self.basedir must be filled in by our parent # self.status must be filled in by our parent @@ -1356,8 +1370,9 @@ self.buildCache.pop(0) def getBuildByNumber(self, number): - if self.currentBuild and self.currentBuild.number == number: - return self.currentBuild + for b in self.currentBuilds: + if b.number == number: + return b for build in self.buildCache: if build.number == number: return build @@ -1387,16 +1402,16 @@ return self.name def getState(self): - return (self.currentBigState, self.currentBuild) + return (self.currentBigState, self.currentBuilds) - def getSlave(self): - return self.status.getSlave(self.slavename) + def getSlaves(self): + return [self.status.getSlave(name) for name in self.slavenames] def getPendingBuilds(self): return self.pendingBuilds - def getCurrentBuild(self): - return self.currentBuild + def getCurrentBuilds(self): + return self.currentBuilds def getLastFinishedBuild(self): b = self.getBuild(-1) @@ -1461,8 +1476,8 @@ ## Builder interface (methods called by the Builder which feeds us) - def setSlavename(self, name): - self.slavename = name + def setSlavenames(self, names): + self.slavenames = names def addEvent(self, text=[], color=None): # this adds a duration event. When it is done, the user should call @@ -1486,7 +1501,7 @@ return e # for consistency, but they really shouldn't touch it def setBigState(self, state): - needToUpdate = state != self.currentBuild + needToUpdate = state != self.currentBigState self.currentBigState = state if needToUpdate: self.publishState() @@ -1525,8 +1540,9 @@ assert s.builder is self # paranoia assert s.number == self.nextBuildNumber - 1 - self.currentBuild = s - self.addBuildToCache(self.currentBuild) + assert s not in self.currentBuilds + self.currentBuilds.append(s) + self.addBuildToCache(s) # now that the BuildStatus is prepared to answer queries, we can # announce the new build to all our watchers @@ -1540,9 +1556,9 @@ s.subscribe(receiver) def _buildFinished(self, s): - assert s is self.currentBuild - self.currentBuild.saveYourself() - self.currentBuild = None + assert s in self.currentBuilds + s.saveYourself() + self.currentBuilds.remove(s) name = self.getName() results = s.getResults() Index: client.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/client.py,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- client.py 31 Aug 2005 01:12:06 -0000 1.23 +++ client.py 14 Oct 2005 19:42:40 -0000 1.24 @@ -72,17 +72,19 @@ return self.b.getName() def remote_getState(self): - state, build = self.b.getState() - return (state, None, makeRemote(build)) # TODO: remove leftover ETA + state, builds = self.b.getState() + return (state, + None, # TODO: remove leftover ETA + [makeRemote(b) for b in builds]) - def remote_getSlave(self): - return IRemote(self.b.getSlave()) + def remote_getSlaves(self): + return [IRemote(s) for s in self.b.getSlaves()] def remote_getLastFinishedBuild(self): return makeRemote(self.b.getLastFinishedBuild()) - def remote_getCurrentBuild(self): - return makeRemote(self.b.getCurrentBuild()) + def remote_getCurrentBuilds(self): + return makeRemote(self.b.getCurrentBuilds()) def remote_getBuild(self, number): return makeRemote(self.b.getBuild(number)) Index: html.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v retrieving revision 1.68 retrieving revision 1.69 diff -u -d -r1.68 -r1.69 --- html.py 2 Sep 2005 15:40:41 -0000 1.68 +++ html.py 14 Oct 2005 19:42:40 -0000 1.69 @@ -303,7 +303,8 @@ % (b.getBuilder().getName(), b.getNumber(), html.escape(b.getReason()))) if b.isFinished(): - data += "

Results:

" + data += "

Buildslave: %s

\n" % html.escape(b.getSlavename()) + data += "

Results:

\n" data += " ".join(b.getText()) + "\n" if b.getTestResults(): url = request.childLink("tests") @@ -382,22 +383,30 @@ def body(self, request): b = self.builder - slave = b.getSlave() + slaves = b.getSlaves() + connected_slaves = [s for s in slaves if s.isConnected()] + data = make_row("Builder:", html.escape(b.getName())) b1 = b.getBuild(-1) if b1 is not None: data += make_row("Current/last build:", str(b1.getNumber())) - if slave.isConnected(): - data += "\nCONNECTED (slave '%s')
\n" % slave.getName() - if slave.getAdmin(): - data += make_row("Admin:", html.escape(slave.getAdmin())) - if slave.getHost(): - data += "Host info:\n" - data += html.PRE(slave.getHost()) - else: - data += "\nNOT CONNECTED (slave '%s')
\n" % slave.getName() + data += "\n
BUILDSLAVES
\n" + data += "
    \n" + for slave in slaves: + data += "
  1. %s: " % html.escape(slave.getName()) + if slave.isConnected(): + data += "CONNECTED\n" + if slave.getAdmin(): + data += make_row("Admin:", html.escape(slave.getAdmin())) + if slave.getHost(): + data += "Host info:\n" + data += html.PRE(slave.getHost()) + else: + data += ("NOT CONNECTED\n") + data += "
  2. \n" + data += "
\n" - if self.control is not None and slave.isConnected(): + if self.control is not None and connected_slaves: forceURL = urllib.quote(request.childLink("force")) data += ( """ @@ -414,7 +423,7 @@ """) % {"forceURL": forceURL} elif self.control is not None: data += """ -

This slave appears to be offline, so it's not possible +

All buildslaves appear to be offline, so it's not possible to force this build to execute at this time.

""" @@ -422,7 +431,7 @@ pingURL = urllib.quote(request.childLink("ping")) data += """
-

To ping a builder, push the 'Ping' button

+

To ping the buildslave(s), push the 'Ping' button

@@ -684,15 +693,16 @@ def getBox(self, status): # getState() returns offline, idle, or building - state, build = self.original.getState() + state, builds = self.original.getState() color = "white" if state == "building": color = "yellow" text = ["building"] - if build: - eta = build.getETA() - if eta: - text.extend(self.formatETA(eta)) + if builds: + for b in builds: + eta = b.getETA() + if eta: + text.extend(self.formatETA(eta)) elif state == "offline": color = "red" text = ["offline"] @@ -1013,7 +1023,7 @@ for b in builders: text = "" color = "#ca88f7" - state, build = b.getState() + state, builds = b.getState() if state != "offline": text += "%s
\n" % state #b.getCurrentBig().text[0] else: Index: words.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/words.py,v retrieving revision 1.40 retrieving revision 1.41 diff -u -d -r1.40 -r1.41 --- words.py 1 Sep 2005 20:53:21 -0000 1.40 +++ words.py 14 Oct 2005 19:42:40 -0000 1.41 @@ -242,20 +242,21 @@ raise UsageError("try 'watch '") which = args[0] b = self.getBuilder(which) - build = b.getCurrentBuild() - if not build: - self.reply(reply, "there is no build currently running") + builds = b.getCurrentBuilds() + if not builds: + self.reply(reply, "there are no builds currently running") return - assert not build.isFinished() - d = build.waitUntilFinished() - d.addCallback(self.buildFinished, reply) - r = "watching build %s #%d until it finishes" \ - % (which, build.getNumber()) - eta = build.getETA() - if eta is not None: - r += " [%s]" % self.convertTime(eta) - r += ".." - self.reply(reply, r) + for build in builds: + assert not build.isFinished() + d = build.waitUntilFinished() + d.addCallback(self.buildFinished, reply) + r = "watching build %s #%d until it finishes" \ + % (which, build.getNumber()) + eta = build.getETA() + if eta is not None: + r += " [%s]" % self.convertTime(eta) + r += ".." + self.reply(reply, r) command_WATCH.usage = "watch - announce the completion of an active build" def buildFinished(self, b, reply): @@ -331,25 +332,27 @@ # find an in-progress build builderstatus = self.getBuilder(which) - buildstatus = builderstatus.getCurrentBuild() - if not buildstatus: + builds = builderstatus.getCurrentBuilds() + if not builds: self.reply(reply, "sorry, no build is currently running") return - num = buildstatus.getNumber() + for build in builds: + num = build.getNumber() - # obtain the BuildControl object - buildcontrol = buildercontrol.getBuild(num) + # obtain the BuildControl object + buildcontrol = buildercontrol.getBuild(num) - # make it stop - buildcontrol.stopBuild(r) + # make it stop + buildcontrol.stopBuild(r) + + self.reply(reply, "build %d interrupted" % num) - self.reply(reply, "build %d interrupted" % num) command_STOP.usage = "stop build - Stop a running build" def emit_status(self, reply, which): b = self.getBuilder(which) str = "%s: " % which - state, build = b.getState() + state, builds = b.getState() str += state if state == "idle": last = b.getLastFinishedBuild() @@ -358,13 +361,15 @@ str += ", last build %s secs ago: %s" % \ (int(util.now() - finished), " ".join(last.getText())) if state == "building": - build = b.getCurrentBuild() - if build: + t = [] + for build in builds: step = build.getCurrentStep() - str += " (%s)" % " ".join(step.getText()) + s = "(%s)" % " ".join(step.getText()) ETA = build.getETA() if ETA is not None: - str += " [ETA %s]" % self.convertTime(ETA) + s += " [ETA %s]" % self.convertTime(ETA) + t.append(s) + str += ", ".join(t) self.reply(reply, str) def emit_last(self, reply, which): From warner at users.sourceforge.net Fri Oct 14 19:42:41 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:41 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test runutils.py,1.2,1.3 test_locks.py,1.1,1.2 test_config.py,1.25,1.26 test_scheduler.py,1.5,1.6 test_run.py,1.34,1.35 test_slaves.py,1.1,1.2 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/buildbot/test Modified Files: runutils.py test_locks.py test_config.py test_scheduler.py test_run.py test_slaves.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: runutils.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- runutils.py 20 Jul 2005 08:08:23 -0000 1.2 +++ runutils.py 14 Oct 2005 19:42:39 -0000 1.3 @@ -16,8 +16,6 @@ class RunMixin: master = None - slave = None - slave2 = None def rmtree(self, d): try: @@ -28,79 +26,77 @@ raise def setUp(self): + self.slaves = {} self.rmtree("basedir") - self.rmtree("slavebase") - self.rmtree("slavebase2") os.mkdir("basedir") self.master = master.BuildMaster("basedir") self.status = self.master.getStatus() self.control = interfaces.IControl(self.master) - def connectSlave(self, builders=["dummy"]): + def connectOneSlave(self, slavename, opts={}): port = self.master.slavePort._port.getHost().port - os.mkdir("slavebase") - slave = MyBuildSlave("localhost", port, "bot1", "sekrit", - "slavebase", keepalive=0, usePTY=1) + self.rmtree("slavebase-%s" % slavename) + os.mkdir("slavebase-%s" % slavename) + slave = MyBuildSlave("localhost", port, slavename, "sekrit", + "slavebase-%s" % slavename, + keepalive=0, usePTY=1, debugOpts=opts) slave.info = {"admin": "one"} - self.slave = slave + self.slaves[slavename] = slave slave.startService() + + def connectSlave(self, builders=["dummy"], slavename="bot1", + opts={}): + # connect buildslave 'slavename' and wait for it to connect to all of + # the given builders dl = [] # initiate call for all of them, before waiting on result, # otherwise we might miss some for b in builders: dl.append(self.master.botmaster.waitUntilBuilderAttached(b)) d = defer.DeferredList(dl) + self.connectOneSlave(slavename, opts) return d - def connectSlaves(self, builders=["dummy"]): - port = self.master.slavePort._port.getHost().port - os.mkdir("slavebase") - slave1 = MyBuildSlave("localhost", port, "bot1", "sekrit", - "slavebase", keepalive=0, usePTY=1) - slave1.info = {"admin": "one"} - self.slave = slave1 - slave1.startService() - - os.mkdir("slavebase2") - slave2 = MyBuildSlave("localhost", port, "bot2", "sekrit", - "slavebase2", keepalive=0, usePTY=1) - slave2.info = {"admin": "one"} - self.slave2 = slave2 - slave2.startService() - + def connectSlaves(self, slavenames, builders): dl = [] # initiate call for all of them, before waiting on result, # otherwise we might miss some for b in builders: dl.append(self.master.botmaster.waitUntilBuilderAttached(b)) d = defer.DeferredList(dl) + for name in slavenames: + self.connectOneSlave(name) return d def connectSlave2(self): + # this takes over for bot1, so it has to share the slavename port = self.master.slavePort._port.getHost().port - os.mkdir("slavebase2") + self.rmtree("slavebase-bot2") + os.mkdir("slavebase-bot2") + # this uses bot1, really slave = MyBuildSlave("localhost", port, "bot1", "sekrit", - "slavebase2", keepalive=0, usePTY=1) + "slavebase-bot2", keepalive=0, usePTY=1) slave.info = {"admin": "two"} - self.slave2 = slave + self.slaves['bot2'] = slave slave.startService() - def connectSlave3(self): + def connectSlaveFastTimeout(self): # this slave has a very fast keepalive timeout port = self.master.slavePort._port.getHost().port - os.mkdir("slavebase") + self.rmtree("slavebase-bot1") + os.mkdir("slavebase-bot1") slave = MyBuildSlave("localhost", port, "bot1", "sekrit", - "slavebase", keepalive=2, usePTY=1, + "slavebase-bot1", keepalive=2, usePTY=1, keepaliveTimeout=1) slave.info = {"admin": "one"} - self.slave = slave + self.slaves['bot1'] = slave slave.startService() d = self.master.botmaster.waitUntilBuilderAttached("dummy") return d def tearDown(self): log.msg("doing tearDown") - d = self.shutdownSlave() + d = self.shutdownAllSlaves() d.addCallback(self._tearDown_1) d.addCallback(self._tearDown_2) return maybeWait(d) @@ -110,52 +106,67 @@ def _tearDown_2(self, res): self.master = None log.msg("tearDown done") + # various forms of slave death - def shutdownSlave(self): + def shutdownAllSlaves(self): # the slave has disconnected normally: they SIGINT'ed it, or it shut # down willingly. This will kill child processes and give them a # chance to finish up. We return a Deferred that will fire when # everything is finished shutting down. - log.msg("doing shutdownSlave") + log.msg("doing shutdownAllSlaves") dl = [] - if self.slave: - dl.append(self.slave.waitUntilDisconnected()) - dl.append(defer.maybeDeferred(self.slave.stopService)) - if self.slave2: - dl.append(self.slave2.waitUntilDisconnected()) - dl.append(defer.maybeDeferred(self.slave2.stopService)) + for slave in self.slaves.values(): + dl.append(slave.waitUntilDisconnected()) + dl.append(defer.maybeDeferred(slave.stopService)) d = defer.DeferredList(dl) - d.addCallback(self._shutdownSlaveDone) + d.addCallback(self._shutdownAllSlavesDone) return d - def _shutdownSlaveDone(self, res): - self.slave = None - self.slave2 = None - return self.master.botmaster.waitUntilBuilderDetached("dummy") + def _shutdownAllSlavesDone(self, res): + for name in self.slaves.keys(): + del self.slaves[name] + return self.master.botmaster.waitUntilBuilderFullyDetached("dummy") + + def shutdownSlave(self, slavename, buildername): + # this slave has disconnected normally: they SIGINT'ed it, or it shut + # down willingly. This will kill child processes and give them a + # chance to finish up. We return a Deferred that will fire when + # everything is finished shutting down, and the given Builder knows + # that the slave has gone away. + + s = self.slaves[slavename] + dl = [self.master.botmaster.waitUntilBuilderDetached(buildername), + s.waitUntilDisconnected()] + d = defer.DeferredList(dl) + d.addCallback(self._shutdownSlave_done, slavename) + s.stopService() + return d + def _shutdownSlave_done(self, res, slavename): + del self.slaves[slavename] def killSlave(self): # the slave has died, its host sent a FIN. The .notifyOnDisconnect # callbacks will terminate the current step, so the build should be # flunked (no further steps should be started). - self.slave.bf.continueTrying = 0 - bot = self.slave.getServiceNamed("bot") + self.slaves['bot1'].bf.continueTrying = 0 + bot = self.slaves['bot1'].getServiceNamed("bot") broker = bot.builders["dummy"].remote.broker broker.transport.loseConnection() - self.slave = None + del self.slaves['bot1'] - def disappearSlave(self): + def disappearSlave(self, slavename="bot1", buildername="dummy"): # the slave's host has vanished off the net, leaving the connection # dangling. This will be detected quickly by app-level keepalives or # a ping, or slowly by TCP timeouts. - # implement this by replacing the slave Broker's .dataReceived method + # simulate this by replacing the slave Broker's .dataReceived method # with one that just throws away all data. def discard(data): pass - bot = self.slave.getServiceNamed("bot") - broker = bot.builders["dummy"].remote.broker + bot = self.slaves[slavename].getServiceNamed("bot") + broker = bot.builders[buildername].remote.broker broker.dataReceived = discard # seal its ears broker.transport.write = discard # and take away its voice Index: test_config.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_config.py,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- test_config.py 20 Jul 2005 05:07:48 -0000 1.25 +++ test_config.py 14 Oct 2005 19:42:39 -0000 1.26 @@ -592,7 +592,7 @@ b = master.botmaster.builders["builder1"] self.failUnless(isinstance(b, Builder)) self.failUnlessEqual(b.name, "builder1") - self.failUnlessEqual(b.slavename, "bot1") + self.failUnlessEqual(b.slavenames, ["bot1"]) self.failUnlessEqual(b.builddir, "workdir") f1 = b.buildFactory self.failUnless(isinstance(f1, BasicBuildFactory)) Index: test_locks.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_locks.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- test_locks.py 19 Jul 2005 23:11:58 -0000 1.1 +++ test_locks.py 14 Oct 2005 19:42:39 -0000 1.2 @@ -63,7 +63,8 @@ req1.events = req2.events = req3.events = self.events = [] d = self.master.loadConfig(config_1) d.addCallback(lambda res: self.master.startService()) - d.addCallback(lambda res: self.connectSlaves(["full1a", "full1b", + d.addCallback(lambda res: self.connectSlaves(["bot1", "bot2"], + ["full1a", "full1b", "full1c", "full1d", "full2a", "full2b"])) return maybeWait(d) Index: test_run.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_run.py,v retrieving revision 1.34 retrieving revision 1.35 diff -u -d -r1.34 -r1.35 --- test_run.py 7 Oct 2005 18:45:42 -0000 1.34 +++ test_run.py 14 Oct 2005 19:42:39 -0000 1.35 @@ -206,8 +206,8 @@ ["testdummy"]) self.s1 = s1 = s.getBuilder("dummy") self.failUnlessEqual(s1.getName(), "dummy") - self.failUnlessEqual(s1.getState(), ("offline", None)) - self.failUnlessEqual(s1.getCurrentBuild(), None) + self.failUnlessEqual(s1.getState(), ("offline", [])) + self.failUnlessEqual(s1.getCurrentBuilds(), []) self.failUnlessEqual(s1.getLastFinishedBuild(), None) self.failUnlessEqual(s1.getBuild(-1), None) #self.failUnlessEqual(s1.getEvent(-1), foo("created")) @@ -364,8 +364,8 @@ self.failUnlessEqual(s.getBuilderNames(), ["dummy", "testdummy"]) self.s1 = s1 = s.getBuilder("dummy") self.failUnlessEqual(s1.getName(), "dummy") - self.failUnlessEqual(s1.getState(), ("offline", None)) - self.failUnlessEqual(s1.getCurrentBuild(), None) + self.failUnlessEqual(s1.getState(), ("offline", [])) + self.failUnlessEqual(s1.getCurrentBuilds(), []) self.failUnlessEqual(s1.getLastFinishedBuild(), None) self.failUnlessEqual(s1.getBuild(-1), None) @@ -374,7 +374,7 @@ return maybeWait(d) def _disconnectSetup_1(self, res): - self.failUnlessEqual(self.s1.getState(), ("idle", None)) + self.failUnlessEqual(self.s1.getState(), ("idle", [])) def verifyDisconnect(self, bs): @@ -399,7 +399,7 @@ def testIdle1(self): # disconnect the slave before the build starts - d = self.shutdownSlave() # dies before it gets started + d = self.shutdownAllSlaves() # dies before it gets started d.addCallback(self._testIdle1_1) return d def _testIdle1_1(self, res): @@ -412,7 +412,7 @@ def testIdle2(self): # now suppose the slave goes missing - self.slave.bf.continueTrying = 0 + self.slaves['bot1'].bf.continueTrying = 0 self.disappearSlave() # forcing a build will work: the build detect that the slave is no @@ -449,7 +449,7 @@ def _testBuild1_1(self, bc): bs = bc.getStatus() # now kill the slave before it gets to start the first step - d = self.shutdownSlave() # dies before it gets started + d = self.shutdownAllSlaves() # dies before it gets started d.addCallback(self._testBuild1_2, bs) return d # TODO: this used to have a 5-second timeout @@ -479,7 +479,7 @@ def _testBuild1_1(self, bc): bs = bc.getStatus() # shutdown the slave while it's running the first step - reactor.callLater(0.5, self.shutdownSlave) + reactor.callLater(0.5, self.shutdownAllSlaves) d = bs.waitUntilFinished() d.addCallback(self._testBuild2_2, bs) @@ -582,7 +582,7 @@ self.failUnlessEqual(res, True) # now, before any build is run, make the slave disappear - self.slave.bf.continueTrying = 0 + self.slaves['bot1'].bf.continueTrying = 0 self.disappearSlave() # at this point, a ping to the slave should timeout @@ -595,13 +595,13 @@ def testDuplicate(self): bc = self.control.getBuilder("dummy") bs = self.status.getBuilder("dummy") - ss = bs.getSlave() + ss = bs.getSlaves()[0] self.failUnless(ss.isConnected()) self.failUnlessEqual(ss.getAdmin(), "one") # now, before any build is run, make the first slave disappear - self.slave.bf.continueTrying = 0 + self.slaves['bot1'].bf.continueTrying = 0 self.disappearSlave() d = self.master.botmaster.waitUntilBuilderDetached("dummy") @@ -638,24 +638,24 @@ self.failUnlessEqual(s.getBuilderNames(), ["dummy", "testdummy"]) self.s1 = s1 = s.getBuilder("dummy") self.failUnlessEqual(s1.getName(), "dummy") - self.failUnlessEqual(s1.getState(), ("offline", None)) - self.failUnlessEqual(s1.getCurrentBuild(), None) + self.failUnlessEqual(s1.getState(), ("offline", [])) + self.failUnlessEqual(s1.getCurrentBuilds(), []) self.failUnlessEqual(s1.getLastFinishedBuild(), None) self.failUnlessEqual(s1.getBuild(-1), None) - d = self.connectSlave3() + d = self.connectSlaveFastTimeout() d.addCallback(self._setup_disconnect2_1) return maybeWait(d) def _setup_disconnect2_1(self, res): - self.failUnlessEqual(self.s1.getState(), ("idle", None)) + self.failUnlessEqual(self.s1.getState(), ("idle", [])) def testSlaveTimeout(self): # now suppose the slave goes missing. We want to find out when it # creates a new Broker, so we reach inside and mark it with the # well-known sigil of impending messy death. - bd = self.slave.getServiceNamed("bot").builders["dummy"] + bd = self.slaves['bot1'].getServiceNamed("bot").builders["dummy"] broker = bd.remote.broker broker.redshirt = 1 @@ -667,7 +667,7 @@ testSlaveTimeout.timeout = 20 def _testSlaveTimeout_1(self, res): - bd = self.slave.getServiceNamed("bot").builders["dummy"] + bd = self.slaves['bot1'].getServiceNamed("bot").builders["dummy"] if not bd.remote or not hasattr(bd.remote.broker, "redshirt"): self.fail("slave disconnected when it shouldn't have") @@ -688,7 +688,7 @@ def _testSlaveTimeout_3(self, res): # make sure it is a new connection (i.e. a new Broker) - bd = self.slave.getServiceNamed("bot").builders["dummy"] + bd = self.slaves['bot1'].getServiceNamed("bot").builders["dummy"] self.failUnless(bd.remote, "hey, slave isn't really connected") self.failIf(hasattr(bd.remote.broker, "redshirt"), "hey, slave's Broker is still marked for death") @@ -706,12 +706,12 @@ return maybeWait(d) def _testChangeBuilddir_1(self, res): - self.bot = bot = self.slave.bot + self.bot = bot = self.slaves['bot1'].bot self.builder = builder = bot.builders.get("dummy") self.failUnless(builder) self.failUnlessEqual(builder.builddir, "dummy") self.failUnlessEqual(builder.basedir, - os.path.join("slavebase", "dummy")) + os.path.join("slavebase-bot1", "dummy")) d = self.master.loadConfig(config_4_newbasedir) d.addCallback(self._testChangeBuilddir_2) @@ -726,7 +726,7 @@ # the basedir should be updated self.failUnlessEqual(builder.builddir, "dummy2") self.failUnlessEqual(builder.basedir, - os.path.join("slavebase", "dummy2")) + os.path.join("slavebase-bot1", "dummy2")) # add a new builder, which causes the basedir list to be reloaded d = self.master.loadConfig(config_4_newbuilder) Index: test_scheduler.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_scheduler.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- test_scheduler.py 31 Aug 2005 01:12:07 -0000 1.5 +++ test_scheduler.py 14 Oct 2005 19:42:39 -0000 1.6 @@ -46,20 +46,6 @@ s1 = self.master.sets[0] self.failUnlessEqual(s1.builderNames, ["a","b"]) - def testPeriodic2(self): - # Twisted-2.0 starts the TimerService right away - # Twisted-1.3 waits one interval before starting it. - # so don't bother asserting anything about it - raise unittest.SkipTest("twisted-1.3 and -2.0 are inconsistent") - self.addScheduler(scheduler.Periodic("hourly", ["a","b"], 3600)) - d = defer.Deferred() - reactor.callLater(1, d.callback, None) - d.addCallback(self._testPeriodic2_1) - return maybeWait(d) - def _testPeriodic2_1(self, res): - # the Periodic scheduler *should* fire right away - self.failUnless(self.master.sets) - def isImportant(self, change): if "important" in change.files: return True Index: test_slaves.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slaves.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- test_slaves.py 19 Jul 2005 23:11:58 -0000 1.1 +++ test_slaves.py 14 Oct 2005 19:42:39 -0000 1.2 @@ -2,16 +2,19 @@ from twisted.trial import unittest from buildbot.twcompat import maybeWait +from twisted.internet import defer, reactor from buildbot.test.runutils import RunMixin - +from buildbot.sourcestamp import SourceStamp +from buildbot.process.base import BuildRequest +from buildbot.status.builder import SUCCESS config_1 = """ from buildbot.process import step, factory s = factory.s BuildmasterConfig = c = {} -c['bots'] = [('bot1', 'sekrit'), ('bot2', 'sekrit')] +c['bots'] = [('bot1', 'sekrit'), ('bot2', 'sekrit'), ('bot3', 'sekrit')] c['sources'] = [] c['schedulers'] = [] c['slavePortnum'] = 0 @@ -20,43 +23,166 @@ f = factory.BuildFactory([s(step.RemoteDummy, timeout=1)]) c['builders'] = [ - {'name': 'b1', 'slavename': 'bot1', 'builddir': 'b1', 'factory': f}, + {'name': 'b1', 'slavenames': ['bot1','bot2','bot3'], + 'builddir': 'b1', 'factory': f}, ] - """ class Slave(RunMixin, unittest.TestCase): - skip = "Not implemented yet" + def setUp(self): RunMixin.setUp(self) self.master.loadConfig(config_1) self.master.startService() d = self.connectSlave(["b1"]) + d.addCallback(lambda res: self.connectSlave(["b1"], "bot2")) return maybeWait(d) - def testClaim(self): - # have three slaves connect for the same builder, make sure all show - # up in the list of known slaves. + def doBuild(self, buildername): + br = BuildRequest("forced", SourceStamp()) + d = self.control.getBuilder(buildername).requestBuild(br) + return d - # run a build, make sure it doesn't freak out. + def testSequence(self): + # make sure both slaves appear in the list. + attached_slaves = [c for c in self.master.botmaster.slaves.values() + if c.slave] + self.failUnlessEqual(len(attached_slaves), 2) + b = self.master.botmaster.builders["b1"] + self.failUnlessEqual(len(b.slaves), 2) + + # since the current scheduling algorithm is simple and does not + # rotate or attempt any sort of load-balancing, two builds in + # sequence should both use the first slave. This may change later if + # we move to a more sophisticated scheme. + + d = self.doBuild("b1") + d.addCallback(self._testSequence_1) + return maybeWait(d) + def _testSequence_1(self, res): + self.failUnlessEqual(res.getResults(), SUCCESS) + self.failUnlessEqual(res.getSlavename(), "bot1") + + d = self.doBuild("b1") + d.addCallback(self._testSequence_2) + return d + def _testSequence_2(self, res): + self.failUnlessEqual(res.getSlavename(), "bot1") + + + def testSimultaneous(self): + # make sure we can actually run two builds at the same time + d1 = self.doBuild("b1") + d2 = self.doBuild("b1") + d1.addCallback(self._testSimultaneous_1, d2) + return maybeWait(d1) + def _testSimultaneous_1(self, res, d2): + self.failUnlessEqual(res.getResults(), SUCCESS) + self.failUnlessEqual(res.getSlavename(), "bot1") + d2.addCallback(self._testSimultaneous_2) + return d2 + def _testSimultaneous_2(self, res): + self.failUnlessEqual(res.getResults(), SUCCESS) + self.failUnlessEqual(res.getSlavename(), "bot2") + def testFallback1(self): + # detach the first slave, verify that a build is run using the second + # slave instead + d = self.shutdownSlave("bot1", "b1") + d.addCallback(self._testFallback1_1) + return maybeWait(d) + def _testFallback1_1(self, res): + attached_slaves = [c for c in self.master.botmaster.slaves.values() + if c.slave] + self.failUnlessEqual(len(attached_slaves), 1) + self.failUnlessEqual(len(self.master.botmaster.builders["b1"].slaves), + 1) + d = self.doBuild("b1") + d.addCallback(self._testFallback1_2) + return d + def _testFallback1_2(self, res): + self.failUnlessEqual(res.getResults(), SUCCESS) + self.failUnlessEqual(res.getSlavename(), "bot2") + + def testFallback2(self): # Disable the first slave, so that a slaveping will timeout. Then # start a build, and verify that the non-failing (second) one is - # claimed for the build, and that the failing one is moved to the - # back of the list. - print "done" + # claimed for the build, and that the failing one is removed from the + # list. + + # reduce the ping time so we'll failover faster + self.master.botmaster.builders["b1"].START_BUILD_TIMEOUT = 1 + self.disappearSlave("bot1", "b1") + d = self.doBuild("b1") + d.addCallback(self._testFallback2_1) + return maybeWait(d) + def _testFallback2_1(self, res): + self.failUnlessEqual(res.getResults(), SUCCESS) + self.failUnlessEqual(res.getSlavename(), "bot2") + b1slaves = self.master.botmaster.builders["b1"].slaves + self.failUnlessEqual(len(b1slaves), 1) + self.failUnlessEqual(b1slaves[0].slave.slavename, "bot2") + + + def notFinished(self, brs): + # utility method + builds = brs.getBuilds() + self.failIf(len(builds) > 1) + if builds: + self.failIf(builds[0].isFinished()) def testDontClaimPingingSlave(self): # have two slaves connect for the same builder. Do something to the # first one so that slavepings are delayed (but do not fail # outright). + timers = [] + self.slaves['bot1'].debugOpts["stallPings"] = (10, timers) + br = BuildRequest("forced", SourceStamp()) + d1 = self.control.getBuilder("b1").requestBuild(br) + s1 = br.status # this is a BuildRequestStatus + # give it a chance to start pinging + d2 = defer.Deferred() + d2.addCallback(self._testDontClaimPingingSlave_1, d1, s1, timers) + reactor.callLater(1, d2.callback, None) + return maybeWait(d2) + def _testDontClaimPingingSlave_1(self, res, d1, s1, timers): + # now the first build is running (waiting on the ping), so start the + # second build. This should claim the second slave, not the first, + # because the first is busy doing the ping. + self.notFinished(s1) + d3 = self.doBuild("b1") + d3.addCallback(self._testDontClaimPingingSlave_2, d1, s1, timers) + return d3 + def _testDontClaimPingingSlave_2(self, res, d1, s1, timers): + self.failUnlessEqual(res.getSlavename(), "bot2") + self.notFinished(s1) + # now let the ping complete + self.failUnlessEqual(len(timers), 1) + timers[0].reset(0) + d1.addCallback(self._testDontClaimPingingSlave_3) + return d1 + def _testDontClaimPingingSlave_3(self, res): + self.failUnlessEqual(res.getSlavename(), "bot1") - # submit a build, which should claim the first slave and send the - # slaveping. While that is (slowly) happening, submit a second build. - # Verify that the second build does not claim the first slave (since - # it is busy doing the slaveping). - pass +class Slave2(RunMixin, unittest.TestCase): + + revision = 0 + + def setUp(self): + RunMixin.setUp(self) + self.master.loadConfig(config_1) + self.master.startService() + + def doBuild(self, buildername, reason="forced"): + # we need to prevent these builds from being merged, so we create + # each of them with a different revision specifier. The revision is + # ignored because our build process does not have a source checkout + # step. + self.revision += 1 + br = BuildRequest(reason, SourceStamp(revision=self.revision)) + d = self.control.getBuilder(buildername).requestBuild(br) + return d def testFirstComeFirstServed(self): # submit three builds, then connect a slave which fails the @@ -64,5 +190,36 @@ # give up, and re-queue the build. Verify that the build gets # re-queued in front of all other builds. This may be tricky, because # the other builds may attempt to claim the just-failed slave. - pass - + + d1 = self.doBuild("b1", "first") + d2 = self.doBuild("b1", "second") + #buildable = self.master.botmaster.builders["b1"].buildable + #print [b.reason for b in buildable] + + # specifically, I want the poor build to get precedence over any + # others that were waiting. To test this, we need more builds than + # slaves. + + # now connect a broken slave. The first build started as soon as it + # connects, so by the time we get to our _1 method, the ill-fated + # build has already started. + d = self.connectSlave(["b1"], opts={"failPingOnce": True}) + d.addCallback(self._testFirstComeFirstServed_1, d1, d2) + return maybeWait(d) + def _testFirstComeFirstServed_1(self, res, d1, d2): + # the master has send the slaveping. When this is received, it will + # fail, causing the master to hang up on the slave. When it + # reconnects, it should find the first build at the front of the + # queue. If we simply wait for both builds to complete, then look at + # the status logs, we should see that the builds ran in the correct + # order. + + d = defer.DeferredList([d1,d2]) + d.addCallback(self._testFirstComeFirstServed_2) + return d + def _testFirstComeFirstServed_2(self, res): + b = self.status.getBuilder("b1") + builds = b.getBuild(0), b.getBuild(1) + reasons = [build.getReason() for build in builds] + self.failUnlessEqual(reasons, ["first", "second"]) + From warner at users.sourceforge.net Fri Oct 14 19:42:42 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:42 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process base.py,1.59,1.60 builder.py,1.31,1.32 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/buildbot/process Modified Files: base.py builder.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: base.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/base.py,v retrieving revision 1.59 retrieving revision 1.60 diff -u -d -r1.59 -r1.60 --- base.py 7 Oct 2005 18:37:21 -0000 1.59 +++ base.py 14 Oct 2005 19:42:40 -0000 1.60 @@ -212,7 +212,7 @@ return files def __repr__(self): - return "" % (self.builder.name) + return "" % (self.builder.name,) def __getstate__(self): d = self.__dict__.copy() @@ -266,6 +266,7 @@ log.msg("%s.startBuild" % self) self.build_status = build_status self.slavebuilder = slavebuilder + self.slavename = slavebuilder.slave.slavename self.locks = [l.getLock(self.slavebuilder) for l in self.locks] self.remote = slavebuilder.remote self.remote.notifyOnDisconnect(self.lostRemote) @@ -509,6 +510,7 @@ self.results = results log.msg(" %s: build finished" % self) + self.build_status.setSlavename(self.slavename) self.build_status.setText(text) self.build_status.setColor(color) self.build_status.setResults(results) Index: builder.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/builder.py,v retrieving revision 1.31 retrieving revision 1.32 diff -u -d -r1.31 -r1.32 --- builder.py 7 Oct 2005 18:37:21 -0000 1.31 +++ builder.py 14 Oct 2005 19:42:40 -0000 1.32 @@ -13,12 +13,19 @@ from buildbot.util import now from buildbot.process import base +(ATTACHING, # slave attached, still checking hostinfo/etc + IDLE, # idle, available for use + PINGING, # build about to start, making sure it is still alive + BUILDING, # build is running + ) = range(4) + class SlaveBuilder(pb.Referenceable): """I am the master-side representative for one of the L{buildbot.slave.bot.SlaveBuilder} objects that lives in a remote buildbot. When a remote builder connects, I query it for command versions and then make it available to any Builds that are ready to run. """ + state = ATTACHING remote = None build = None @@ -60,6 +67,8 @@ return why def detached(self): + log.msg("Buildslave %s detached from %s" % (self.slave.slavename, + self.builder.name)) self.slave = None self.remote = None self.remoteCommands = None @@ -117,7 +126,11 @@ self.running = True log.msg("sending ping") self.d = defer.Deferred() - remote.callRemote("print", "ping").addBoth(self._pong) + # TODO: add a distinct 'ping' command on the slave.. using 'print' + # for this purpose is kind of silly. + remote.callRemote("print", "ping").addCallbacks(self._pong, + self._ping_failed, + errbackArgs=(remote,)) # We use either our own timeout or the (long) TCP timeout to detect # silently-missing slaves. This might happen because of a NAT @@ -135,20 +148,31 @@ remote.broker.transport.loseConnection() # the forcibly-lost connection will now cause the ping to fail - def _pong(self, res): + def _stopTimer(self): if not self.running: return self.running = False - log.msg("ping finished") if self.timer: self.timer.cancel() del self.timer - if isinstance(res, failure.Failure): - self.d.callback(False) - else: - self.d.callback(True) + def _pong(self, res): + log.msg("ping finished: success") + self._stopTimer() + self.d.callback(True) + + def _ping_failed(self, res, remote): + log.msg("ping finished: failure") + self._stopTimer() + # the slave has some sort of internal error, disconnect them. If we + # don't, we'll requeue a build and ping them again right away, + # creating a nasty loop. + remote.broker.transport.loseConnection() + # TODO: except, if they actually did manage to get this far, they'll + # probably reconnect right away, and we'll do this game again. Maybe + # it would be better to leave them in the PINGING state. + self.d.callback(False) class Builder(pb.Referenceable): @@ -172,9 +196,9 @@ discard it. I maintain a list of available SlaveBuilders, one for each connected - slave that the C{slavename} parameter says we can use. Some of these will - be idle, some of them will be busy running builds for me. If there are - multiple slaves, I can run multiple builds at once. + slave that the C{slavenames} parameter says we can use. Some of these + will be idle, some of them will be busy running builds for me. If there + are multiple slaves, I can run multiple builds at once. I also manage forced builds, progress expectation (ETA) management, and some status delivery chores. @@ -201,11 +225,15 @@ @type setup: dict @param setup: builder setup data, as stored in BuildmasterConfig['builders']. Contains name, - slavename, builddir, factory, locks. + slavename(s), builddir, factory, locks. @type builder_status: L{buildbot.status.builder.BuilderStatus} """ self.name = setup['name'] - self.slavename = setup['slavename'] + self.slavenames = [] + if setup.has_key('slavename'): + self.slavenames.append(setup['slavename']) + if setup.has_key('slavenames'): + self.slavenames.extend(setup['slavenames']) self.builddir = setup['builddir'] self.buildFactory = setup['factory'] self.locks = setup.get("locks", []) @@ -218,26 +246,34 @@ self.buildable = [] self.building = [] - # buildslaves at our disposal. This maps SlaveBuilder instances to - # state, where state is one of "attaching", "idle", "pinging", - # "busy". "pinging" is used when a Build is about to start, to make - # sure that they're still alive. - self.slaves = {} + # buildslaves which have connected but which are not yet available. + # These are always in the ATTACHING state. + self.attaching_slaves = [] + + # buildslaves at our disposal. Each SlaveBuilder instance has a + # .state that is IDLE, PINGING, or BUILDING. "PINGING" is used when a + # Build is about to start, to make sure that they're still alive. + self.slaves = [] self.builder_status = builder_status - self.builder_status.setSlavename(self.slavename) + self.builder_status.setSlavenames(self.slavenames) # for testing, to help synchronize tests - self.watchers = {'attach': [], 'detach': [], 'idle': []} + self.watchers = {'attach': [], 'detach': [], 'detach_all': [], + 'idle': []} def setBotmaster(self, botmaster): self.botmaster = botmaster def compareToSetup(self, setup): diffs = [] - if setup['slavename'] != self.slavename: - diffs.append('slavename changed from %s to %s' \ - % (self.slavename, setup['slavename'])) + setup_slavenames = [] + if setup.has_key('slavename'): + setup_slavenames.append(setup['slavename']) + setup_slavenames.extend(setup.get('slavenames', [])) + if setup_slavenames != self.slavenames: + diffs.append('slavenames changed from %s to %s' \ + % (self.slavenames, setup_slavenames)) if setup['builddir'] != self.builddir: diffs.append('builddir changed from %s to %s' \ % (self.builddir, setup['builddir'])) @@ -279,7 +315,7 @@ def __setstate__(self, d): self.__dict__ = d self.building = [] - self.slaves = {} + self.slaves = [] def fireTestEvent(self, name, with=None): if with is None: @@ -287,7 +323,7 @@ watchers = self.watchers[name] self.watchers[name] = [] for w in watchers: - w.callback(with) + reactor.callLater(0, w.callback, with) def attached(self, slave, remote, commands): """This is invoked by the BotPerspective when the self.slavename bot @@ -305,7 +341,7 @@ @return: a Deferred that fires (with 'self') when the slave-side builder is fully attached and ready to accept commands. """ - for s in self.slaves.keys(): + for s in self.attaching_slaves + self.slaves: if s.slave == slave: # already attached to them. This is fairly common, since # attached() gets called each time we receive the builder @@ -319,14 +355,14 @@ # Therefore, when we see that we're already attached, we can # just ignore it. TODO: build a diagram of the state # transitions here, I'm concerned about sb.attached() failing - # and leaving self.slaves[sb] stuck at 'attaching', and about + # and leaving sb.state stuck at 'ATTACHING', and about # the detached() message arriving while there's some # transition pending such that the response to the transition - # re-vivifies self.slaves[sb] + # re-vivifies sb return defer.succeed(self) sb = SlaveBuilder(self) - self.slaves[sb] = "attaching" + self.attaching_slaves.append(sb) d = sb.attached(slave, remote, commands) d.addCallback(self._attached) d.addErrback(self._not_attached, slave) @@ -335,7 +371,9 @@ def _attached(self, sb): # TODO: make this .addSlaveEvent(slave.slavename, ['connect']) ? self.builder_status.addPointEvent(['connect', sb.slave.slavename]) - self.slaves[sb] = "idle" + sb.state = IDLE + self.attaching_slaves.remove(sb) + self.slaves.append(sb) self.maybeStartBuild() self.fireTestEvent('attach') @@ -354,10 +392,17 @@ def detached(self, slave): """This is called when the connection to the bot is lost.""" log.msg("%s.detached" % self, slave.slavename) - for sb in self.slaves.keys(): + for sb in self.attaching_slaves + self.slaves: if sb.slave == slave: break - if self.slaves[sb] == "busy": + else: + log.msg("WEIRD: Builder.detached(%s) (%s)" + " not in attaching_slaves(%s)" + " or slaves(%s)" % (slave, slave.slavename, + self.attaching_slaves, + self.slaves)) + return + if sb.state == BUILDING: # the Build's .lostRemote method (invoked by a notifyOnDisconnect # handler) will cause the Build to be stopped, probably right # after the notifyOnDisconnect that invoked us finishes running. @@ -366,13 +411,18 @@ #self.retryBuild(sb.build) pass - del self.slaves[sb] + if sb in self.attaching_slaves: + self.attaching_slaves.remove(sb) + if sb in self.slaves: + self.slaves.remove(sb) # TODO: make this .addSlaveEvent? self.builder_status.addPointEvent(['disconnect', slave.slavename]) sb.detached() # inform the SlaveBuilder that their slave went away self.updateBigStatus() self.fireTestEvent('detach') + if not self.slaves: + self.fireTestEvent('detach_all') def updateBigStatus(self): if not self.slaves: @@ -388,14 +438,15 @@ if not self.buildable: self.updateBigStatus() return # nothing to do - idle_slaves = [sb for sb in self.slaves.keys() - if self.slaves[sb] == "idle"] - if not idle_slaves: + # find the first idle slave + for sb in self.slaves: + if sb.state == IDLE: + break + else: log.msg("%s: want to start build, but we don't have a remote" % self) self.updateBigStatus() return - sb = idle_slaves[0] # there is something to build, and there is a slave on which to build # it. Grab the oldest request, see if we can merge it with anything @@ -431,8 +482,9 @@ self.building.append(build) - # claim the slave - self.slaves[sb] = "pinging" + # claim the slave. TODO: consider moving changes to sb.state inside + # SlaveBuilder.. that would be cleaner. + sb.state = PINGING sb.startBuild(build) self.updateBigStatus() @@ -451,7 +503,7 @@ if not res: return self._startBuildFailed("slave ping failed", build, sb) # The buildslave is ready to go. - self.slaves[sb] = "building" + sb.state = BUILDING d = sb.remote.callRemote("startBuild") d.addCallbacks(self._startBuild_2, self._startBuildFailed, callbackArgs=(build,sb), errbackArgs=(build,sb)) @@ -479,14 +531,7 @@ "remote_startBuild failed: %s" % (build, why)) # release the slave sb.finishBuild() - if sb in self.slaves: - self.slaves[sb] = "idle" - else: - # if the startBuild message failed because we lost the slave, our - # detacted() method will have already fired, removing the - # SlaveBuilder from self.slaves . This test is here to make sure - # we don't re-create the old self.slaves[sb] entry. - pass + sb.state = IDLE log.msg("re-queueing the BuildRequest") self.building.remove(build) @@ -509,8 +554,7 @@ # release the slave sb.finishBuild() - if sb in self.slaves: - self.slaves[sb] = "idle" + sb.state = IDLE # otherwise the slave probably got removed in detach() self.building.remove(build) @@ -561,12 +605,17 @@ warnings.warn("Please use BuilderControl.requestBuild instead", category=DeprecationWarning, stacklevel=1) - idle_slaves = [sb for sb in self.original.slaves - if self.original.slaves[sb] == "idle"] - if not idle_slaves: + + # see if there is an idle slave, so we can emit an appropriate error + # message + for sb in self.original.slaves: + if sb.state == IDLE: + break + else: if self.original.building: raise interfaces.BuilderInUseError("All slaves are in use") raise interfaces.NoSlaveError("There are no slaves connected") + req = base.BuildRequest(reason, sourcestamp.SourceStamp()) self.requestBuild(req) # this is a hack that fires the Deferred for the first build and @@ -587,7 +636,13 @@ return w.wait() def requestBuild(self, req): + """Submit a BuildRequest to this Builder. Returns a Deferred that + fires when the BuildRequest finishes, the same as doing + req.waitUntilFinished . This Deferred will fire with an + L{buildbot.interfaces.IBuildStatus} instance.""" + d = req.waitUntilFinished() self.original.submitBuildRequest(req) + return d def getPendingBuilds(self): # return IBuildRequestControl objects @@ -604,8 +659,17 @@ self.original.builder_status.addPointEvent(["ping", "no slave"], "red") return defer.succeed(False) # interfaces.NoSlaveError - d = self.original.slaves.keys()[0].ping(timeout, - self.original.builder_status) + dl = [] + for s in self.original.slaves: + dl.append(s.ping(timeout, self.original.builder_status)) + d = defer.DeferredList(dl) + d.addCallback(self._gatherPingResults) return d + def _gatherPingResults(self, res): + for ignored,success in res: + if not success: + return False + return True + components.registerAdapter(BuilderControl, Builder, interfaces.IBuilderControl) From warner at users.sourceforge.net Fri Oct 14 19:42:42 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:42 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.22,1.23 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/docs Modified Files: buildbot.texinfo Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- buildbot.texinfo 2 Sep 2005 15:31:28 -0000 1.22 +++ buildbot.texinfo 14 Oct 2005 19:42:40 -0000 1.23 @@ -1839,6 +1839,22 @@ @code{slavename} must appear in the @code{c['bots']} list. Each buildslave can accomodate multiple Builders. + at item slavenames +If you provide @code{slavenames} instead of @code{slavename}, you can +give a list of buildslaves which are capable of running this Builder. +If multiple buildslaves are connected for any given Builder, it will +be able to run multiple builds at the same time (one per slave). This +can also provide a level of redundancy: in case one slave goes +offline, the others can still keep the Builder working. + +If you use this feature, it is important to make sure that the +buildslaves are all, in fact, capable of running the given build. The +slave hosts should be configured similarly, otherwise you will spend a +lot of time trying (unsuccessfully) to reproduce a failure that only +occurs on some of the buildslaves and not the others. Different +platforms, operating systems, versions of major programs or libraries, +all these things mean you should use separate Builders. + @item builddir This specifies the name of a subdirectory (under the base directory) in which everything related to this builder will be placed. On the From warner at users.sourceforge.net Fri Oct 14 19:42:42 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:42 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.508,1.509 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.508 retrieving revision 1.509 diff -u -d -r1.508 -r1.509 --- ChangeLog 14 Oct 2005 19:32:55 -0000 1.508 +++ ChangeLog 14 Oct 2005 19:42:40 -0000 1.509 @@ -1,3 +1,51 @@ +2005-10-13 Brian Warner + + * buildbot/status/words.py (IrcStatusBot.command_WATCH): fix typo + + * lots: implement multiple slaves per Builder, which means multiple + current builds per Builder. Some highlights: + * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple + of (state,currentBuilds) instead of (state,currentBuild) + (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() + (IBuildStatus.getSlavename): new method, so you can tell which + slave got used. This only gets set when the build completes. + (IBuildRequestStatus.getBuilds): new method + + * buildbot/process/builder.py (SlaveBuilder): add a .state + attribute to track things like ATTACHING and IDLE and BUILDING, + instead of.. + (Builder): .. the .slaves attribute here, which has been turned + into a simple list of available slaves. Added a separate + attaching_slaves list to track ones that are not yet ready for + builds. + (Builder.fireTestEvent): put off the test-event callback for a + reactor turn, to make tests a bit more consistent. + (Ping): cleaned up the slaveping a bit, now it disconnects if the + ping fails due to an exception. This needs work, I'm worried that + a code error could lead to a constantly re-connecting slave. + Especially since I'm trying to move to a distinct remote_ping + method, separate from the remote_print that we currently use. + (BuilderControl.requestBuild): return a convenience Deferred that + provides an IBuildStatus when the build finishes. + (BuilderControl.ping): ping all connected slaves, only return True + if they all respond. + + * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to + reconnect when we shut down. + + * buildbot/status/builder.py: implement new methods, convert + one-build-at-a-time methods to handle multiple builds + * buildbot/status/*.py: do the same in all default status targets + * buildbot/status/html.py: report the build's slavename in the + per-Build page, report all buildslaves on the per-Builder page + + * buildbot/test/test_run.py: update/create tests + * buildbot/test/test_slaves.py: same + * buildbot/test/test_scheduler.py: remove stale test + + * docs/buildbot.texinfo: document the new builder-specification + 'slavenames' parameter + 2005-10-12 Brian Warner * buildbot/buildset.py (BuildSet): fix bug where BuildSet did not From warner at users.sourceforge.net Fri Oct 14 19:42:42 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:42:42 +0000 Subject: [Buildbot-commits] buildbot/buildbot/clients gtkPanes.py,1.8,1.9 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/clients In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/buildbot/clients Modified Files: gtkPanes.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326 Creator: Brian Warner implement multiple slaves per Builder, allowing concurrent Builds * lots: implement multiple slaves per Builder, which means multiple current builds per Builder. Some highlights: * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple of (state,currentBuilds) instead of (state,currentBuild) (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() (IBuildStatus.getSlavename): new method, so you can tell which slave got used. This only gets set when the build completes. (IBuildRequestStatus.getBuilds): new method * buildbot/process/builder.py (SlaveBuilder): add a .state attribute to track things like ATTACHING and IDLE and BUILDING, instead of.. (Builder): .. the .slaves attribute here, which has been turned into a simple list of available slaves. Added a separate attaching_slaves list to track ones that are not yet ready for builds. (Builder.fireTestEvent): put off the test-event callback for a reactor turn, to make tests a bit more consistent. (Ping): cleaned up the slaveping a bit, now it disconnects if the ping fails due to an exception. This needs work, I'm worried that a code error could lead to a constantly re-connecting slave. Especially since I'm trying to move to a distinct remote_ping method, separate from the remote_print that we currently use. (BuilderControl.requestBuild): return a convenience Deferred that provides an IBuildStatus when the build finishes. (BuilderControl.ping): ping all connected slaves, only return True if they all respond. * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to reconnect when we shut down. * buildbot/status/builder.py: implement new methods, convert one-build-at-a-time methods to handle multiple builds * buildbot/status/*.py: do the same in all default status targets * buildbot/status/html.py: report the build's slavename in the per-Build page, report all buildslaves on the per-Builder page * buildbot/test/test_run.py: update/create tests * buildbot/test/test_slaves.py: same * buildbot/test/test_scheduler.py: remove stale test * docs/buildbot.texinfo: document the new builder-specification 'slavenames' parameter Index: gtkPanes.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/clients/gtkPanes.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- gtkPanes.py 23 Apr 2005 10:37:00 -0000 1.8 +++ gtkPanes.py 14 Oct 2005 19:42:40 -0000 1.9 @@ -291,8 +291,10 @@ def getState(self): self.ref.callRemote("getState").addCallback(self.gotState) def gotState(self, res): - state, ETA, build = res + state, ETA, builds = res # state is one of: offline, idle, waiting, interlocked, building + # TODO: ETA is going away, you have to look inside the builds to get + # that value currentmap = {"offline": "red", "idle": "white", "waiting": "yellow", From warner at users.sourceforge.net Fri Oct 14 19:48:00 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:48:00 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_changes.py,1.5,1.6 test_config.py,1.26,1.27 test_status.py,1.23,1.24 test_control.py,1.8,1.9 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1561/buildbot/test Modified Files: test_changes.py test_config.py test_status.py test_control.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-328 Creator: Brian Warner remove almost all remaining uses of deferredResult from unit tests * buildbot/test/test_changes.py: remove use of deferredResult * buildbot/test/test_config.py: same * buildbot/test/test_control.py: same * buildbot/test/test_status.py: same * buildbot/test/test_vc.py: this is the only remaining use, since it gets used at module level. This needs to be replaced by some sort of class-level run-once routine. Index: test_changes.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_changes.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- test_changes.py 19 Jul 2005 23:11:58 -0000 1.5 +++ test_changes.py 14 Oct 2005 19:47:58 -0000 1.6 @@ -1,11 +1,11 @@ # -*- test-case-name: buildbot.test.test_changes -*- from twisted.trial import unittest -dr = unittest.deferredResult from twisted.internet import defer, reactor from twisted.python import log from buildbot import master +from buildbot.twcompat import maybeWait from buildbot.changes import pb from buildbot.scripts import runner @@ -78,50 +78,61 @@ self.master = master.BuildMaster(".") def tearDown(self): d = defer.maybeDeferred(self.master.stopService) - dr(d) # TODO: something in Twisted-2.0.0 (and probably 2.0.1) doesn't shut # down the Broker listening socket when it's supposed to. # Twisted-1.3.0, and current SVN (which will be post-2.0.1) are ok. # This iterate() is a quick hack to deal with the problem. I need to # investigate more thoroughly and find a better solution. - reactor.iterate() + d.addCallback(self.stall, 0.1) + return maybeWait(d) + + def stall(self, res, timeout): + d = defer.Deferred() + reactor.callLater(timeout, d.callback, res) + return d def testSender(self): - d = self.master.loadConfig(config_empty) - dr(d) + self.master.loadConfig(config_empty) self.master.startService() # TODO: BuildMaster.loadChanges replaces the change_svc object, so we # have to load it twice. Clean this up. d = self.master.loadConfig(config_sender) - dr(d) + d.addCallback(self._testSender_1) + return maybeWait(d) - cm = self.master.change_svc - s1 = list(cm)[0] + def _testSender_1(self, res): + self.cm = cm = self.master.change_svc + s1 = list(self.cm)[0] port = self.master.slavePort._port.getHost().port - options = {'username': "alice", - 'master': "localhost:%d" % port, - 'files': ["foo.c"], - } + self.options = {'username': "alice", + 'master': "localhost:%d" % port, + 'files': ["foo.c"], + } - d = runner.sendchange(options) - dr(d) + d = runner.sendchange(self.options) + d.addCallback(self._testSender_2) + return d + def _testSender_2(self, res): # now check that the change was received - self.failUnlessEqual(len(cm.changes), 1) - c = cm.changes.pop() + self.failUnlessEqual(len(self.cm.changes), 1) + c = self.cm.changes.pop() self.failUnlessEqual(c.who, "alice") self.failUnlessEqual(c.files, ["foo.c"]) self.failUnlessEqual(c.comments, "") self.failUnlessEqual(c.revision, None) - options['revision'] = "r123" - options['comments'] = "test change" + self.options['revision'] = "r123" + self.options['comments'] = "test change" - d = runner.sendchange(options) - dr(d) - self.failUnlessEqual(len(cm.changes), 1) - c = cm.changes.pop() + d = runner.sendchange(self.options) + d.addCallback(self._testSender_3) + return d + + def _testSender_3(self, res): + self.failUnlessEqual(len(self.cm.changes), 1) + c = self.cm.changes.pop() self.failUnlessEqual(c.who, "alice") self.failUnlessEqual(c.files, ["foo.c"]) self.failUnlessEqual(c.comments, "test change") @@ -132,27 +143,33 @@ f = open(logfile, "wt") f.write("longer test change") f.close() - options['comments'] = None - options['logfile'] = logfile + self.options['comments'] = None + self.options['logfile'] = logfile - d = runner.sendchange(options) - dr(d) - self.failUnlessEqual(len(cm.changes), 1) - c = cm.changes.pop() + d = runner.sendchange(self.options) + d.addCallback(self._testSender_4) + return d + + def _testSender_4(self, res): + self.failUnlessEqual(len(self.cm.changes), 1) + c = self.cm.changes.pop() self.failUnlessEqual(c.who, "alice") self.failUnlessEqual(c.files, ["foo.c"]) self.failUnlessEqual(c.comments, "longer test change") self.failUnlessEqual(c.revision, "r123") # make sure that numeric revisions work too - options['logfile'] = None - del options['revision'] - options['revision_number'] = 42 + self.options['logfile'] = None + del self.options['revision'] + self.options['revision_number'] = 42 - d = runner.sendchange(options) - dr(d) - self.failUnlessEqual(len(cm.changes), 1) - c = cm.changes.pop() + d = runner.sendchange(self.options) + d.addCallback(self._testSender_5) + return d + + def _testSender_5(self, res): + self.failUnlessEqual(len(self.cm.changes), 1) + c = self.cm.changes.pop() self.failUnlessEqual(c.who, "alice") self.failUnlessEqual(c.files, ["foo.c"]) self.failUnlessEqual(c.comments, "") Index: test_config.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_config.py,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- test_config.py 14 Oct 2005 19:42:39 -0000 1.26 +++ test_config.py 14 Oct 2005 19:47:58 -0000 1.27 @@ -4,7 +4,6 @@ import os, os.path from twisted.trial import unittest -dr = unittest.deferredResult from twisted.python import components, failure from twisted.internet import defer @@ -452,6 +451,7 @@ self.failUnlessEqual(master.checker.users, {"change": "changepw"}) + def testSources(self): if not cvstoys: raise unittest.SkipTest("this test needs CVSToys installed") @@ -469,27 +469,36 @@ """ d = master.loadConfig(sourcesCfg) - dr(d) - self.failUnlessEqual(len(list(master.change_svc)), 1) - s1 = list(master.change_svc)[0] + d.addCallback(self._testSources_1) + return maybeWait(d) + + def _testSources_1(self, res): + self.failUnlessEqual(len(list(self.buildmaster.change_svc)), 1) + s1 = list(self.buildmaster.change_svc)[0] self.failUnless(isinstance(s1, FreshCVSSource)) self.failUnlessEqual(s1.where, ("cvs.example.com", 1000)) self.failUnlessEqual(s1.prefix, "Prefix/") - self.failUnlessEqual(s1, list(master.change_svc)[0]) + self.failUnlessEqual(s1, list(self.buildmaster.change_svc)[0]) self.failUnless(s1.parent) # verify that unchanged sources are not interrupted - d = master.loadConfig(sourcesCfg) - dr(d) - self.failUnlessEqual(len(list(master.change_svc)), 1) - s2 = list(master.change_svc)[0] + d = self.buildmaster.loadConfig(sourcesCfg) + d.addCallback(self._testSources_2) + return d + + def _testSources_2(self, res): + self.failUnlessEqual(len(list(self.buildmaster.change_svc)), 1) + s2 = list(self.buildmaster.change_svc)[0] self.failUnlessIdentical(s1, s2) self.failUnless(s1.parent) # make sure we can get rid of the sources too - d = master.loadConfig(emptyCfg) - dr(d) - self.failUnlessEqual(list(master.change_svc), []) + d = self.buildmaster.loadConfig(emptyCfg) + d.addCallback(self._testSources_3) + return d + + def _testSources_3(self, res): + self.failUnlessEqual(list(self.buildmaster.change_svc), []) def shouldBeFailure(self, res, *expected): self.failUnless(isinstance(res, failure.Failure), @@ -664,89 +673,103 @@ master = self.buildmaster master.loadChanges() d = master.loadConfig(emptyCfg) - dr(d) - self.checkIRC(master, {}) - - d = master.loadConfig(ircCfg1) - dr(d) - expected = {'irc.us.freenode.net': ('buildbot', ['twisted'])} - self.checkIRC(master, expected) - - d = master.loadConfig(ircCfg2) - dr(d) - expected = {'irc.us.freenode.net': ('buildbot', ['twisted']), - 'irc.example.com': ('otherbot', ['chan1', 'chan2'])} - self.checkIRC(master, expected) - - d = master.loadConfig(ircCfg3) - dr(d) - expected = {'irc.us.freenode.net': ('buildbot', ['knotted'])} - self.checkIRC(master, expected) - - d = master.loadConfig(ircCfg1) - dr(d) - expected = {'irc.us.freenode.net': ('buildbot', ['twisted'])} - self.checkIRC(master, expected) + e1 = {} + d.addCallback(lambda res: self.checkIRC(master, e1)) + d.addCallback(lambda res: master.loadConfig(ircCfg1)) + e2 = {'irc.us.freenode.net': ('buildbot', ['twisted'])} + d.addCallback(lambda res: self.checkIRC(master, e2)) + d.addCallback(lambda res: master.loadConfig(ircCfg2)) + e3 = {'irc.us.freenode.net': ('buildbot', ['twisted']), + 'irc.example.com': ('otherbot', ['chan1', 'chan2'])} + d.addCallback(lambda res: self.checkIRC(master, e3)) + d.addCallback(lambda res: master.loadConfig(ircCfg3)) + e4 = {'irc.us.freenode.net': ('buildbot', ['knotted'])} + d.addCallback(lambda res: self.checkIRC(master, e4)) + d.addCallback(lambda res: master.loadConfig(ircCfg1)) + e5 = {'irc.us.freenode.net': ('buildbot', ['twisted'])} + d.addCallback(lambda res: self.checkIRC(master, e5)) + return maybeWait(d) def testWebPortnum(self): master = self.buildmaster master.loadChanges() d = master.loadConfig(webCfg1) - dr(d) - ports = self.checkPorts(master, [(9999, pb.PBServerFactory), - (9980, Site)]) + d.addCallback(self._testWebPortnum_1) + return maybeWait(d) + def _testWebPortnum_1(self, res): + ports = self.checkPorts(self.buildmaster, [(9999, pb.PBServerFactory), + (9980, Site)]) p = ports[1] - d = master.loadConfig(webCfg1) # nothing should be changed - dr(d) - ports = self.checkPorts(master, [(9999, pb.PBServerFactory), - (9980, Site)]) + d = self.buildmaster.loadConfig(webCfg1) # nothing should be changed + d.addCallback(self._testWebPortnum_2, p) + return d + def _testWebPortnum_2(self, res, p): + ports = self.checkPorts(self.buildmaster, [(9999, pb.PBServerFactory), + (9980, Site)]) self.failUnlessIdentical(p, ports[1], "web port was changed even though " + \ "configuration was not") - d = master.loadConfig(webCfg2) # changes to 9981 - dr(d) - ports = self.checkPorts(master, [(9999, pb.PBServerFactory), - (9981, Site)]) + d = self.buildmaster.loadConfig(webCfg2) # changes to 9981 + d.addCallback(self._testWebPortnum_3, p) + return d + def _testWebPortnum_3(self, res, p): + ports = self.checkPorts(self.buildmaster, [(9999, pb.PBServerFactory), + (9981, Site)]) self.failIf(p is ports[1], "web port was unchanged but configuration was changed") - d = master.loadConfig(emptyCfg) - dr(d) - self.checkPorts(master, [(9999, pb.PBServerFactory)]) + d = self.buildmaster.loadConfig(emptyCfg) + d.addCallback(lambda res: + self.checkPorts(self.buildmaster, + [(9999, pb.PBServerFactory)])) + return d def testWebPathname(self): master = self.buildmaster master.loadChanges() d = master.loadConfig(webNameCfg1) - dr(d) - self.checkPorts(master, [(9999, pb.PBServerFactory), - ('~/.twistd-web-pb', pb.PBServerFactory)]) - unixports = self.UNIXports(master) + d.addCallback(self._testWebPathname_1) + return maybeWait(d) + def _testWebPathname_1(self, res): + self.checkPorts(self.buildmaster, + [(9999, pb.PBServerFactory), + ('~/.twistd-web-pb', pb.PBServerFactory)]) + unixports = self.UNIXports(self.buildmaster) f = unixports[0].args[1] self.failUnless(isinstance(f.root, ResourcePublisher)) - d = master.loadConfig(webNameCfg1) # nothing should be changed - dr(d) - self.checkPorts(master, [(9999, pb.PBServerFactory), - ('~/.twistd-web-pb', pb.PBServerFactory)]) - self.failUnlessIdentical(f, self.UNIXports(master)[0].args[1], + d = self.buildmaster.loadConfig(webNameCfg1) + # nothing should be changed + d.addCallback(self._testWebPathname_2, f) + return d + def _testWebPathname_2(self, res, f): + self.checkPorts(self.buildmaster, + [(9999, pb.PBServerFactory), + ('~/.twistd-web-pb', pb.PBServerFactory)]) + self.failUnlessIdentical(f, + self.UNIXports(self.buildmaster)[0].args[1], "web factory was changed even though " + \ "configuration was not") - d = master.loadConfig(webNameCfg2) - dr(d) - self.checkPorts(master, [(9999, pb.PBServerFactory), - ('bar.socket', pb.PBServerFactory)]) - self.failIf(f is self.UNIXports(master)[0].args[1], + d = self.buildmaster.loadConfig(webNameCfg2) + d.addCallback(self._testWebPathname_3, f) + return d + def _testWebPathname_3(self, res, f): + self.checkPorts(self.buildmaster, + [(9999, pb.PBServerFactory), + ('bar.socket', pb.PBServerFactory)]) + self.failIf(f is self.UNIXports(self.buildmaster)[0].args[1], "web factory was unchanged but configuration was changed") - d = master.loadConfig(emptyCfg) - dr(d) - self.checkPorts(master, [(9999, pb.PBServerFactory)]) + d = self.buildmaster.loadConfig(emptyCfg) + d.addCallback(lambda res: + self.checkPorts(self.buildmaster, + [(9999, pb.PBServerFactory)])) + return d def testDebugPassword(self): master = self.buildmaster Index: test_control.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_control.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- test_control.py 17 Aug 2005 02:15:38 -0000 1.8 +++ test_control.py 14 Oct 2005 19:47:58 -0000 1.9 @@ -3,7 +3,6 @@ import sys, os, signal, shutil, time, errno from twisted.trial import unittest -dr = unittest.deferredResult from twisted.internet import defer, reactor from buildbot import master, interfaces @@ -80,18 +79,7 @@ self.slave = slave slave.startService() d = self.master.botmaster.waitUntilBuilderAttached("force") - dr(d) - - def loadConfig(self, config): - # reloading the config file causes a new 'listDirs' command to be - # sent to the slave. To synchronize on this properly, it is easiest - # to stop and restart the slave. - if self.slave: - d = self.master.botmaster.waitUntilBuilderDetached("force") - dr(defer.maybeDeferred(self.slave.stopService)) - dr(d) - self.master.loadConfig(config) - self.connectSlave() + return d def tearDown(self): dl = [] @@ -107,23 +95,25 @@ # test is scheduled to be removed soon m = self.master m.loadConfig(config) - m.readConfig = True m.startService() - self.connectSlave() + d = self.connectSlave() + d.addCallback(self._testForce_1) + return maybeWait(d) - c = interfaces.IControl(m) + def _testForce_1(self, res): + c = interfaces.IControl(self.master) builder_control = c.getBuilder("force") d = builder_control.forceBuild("bob", "I was bored") - d.addCallback(self._testForce_1) - return maybeWait(d) + d.addCallback(self._testForce_2) + return d - def _testForce_1(self, build_control): + def _testForce_2(self, build_control): self.failUnless(providedBy(build_control, interfaces.IBuildControl)) d = build_control.getStatus().waitUntilFinished() - d.addCallback(self._testForce_2) + d.addCallback(self._testForce_3) return d - def _testForce_2(self, bs): + def _testForce_3(self, bs): self.failUnless(providedBy(bs, interfaces.IBuildStatus)) self.failUnless(bs.isFinished()) self.failUnlessEqual(bs.getResults(), SUCCESS) @@ -134,15 +124,17 @@ def testRequest(self): m = self.master m.loadConfig(config) - m.readConfig = True m.startService() - self.connectSlave() - - c = interfaces.IControl(m) + d = self.connectSlave() + d.addCallback(self._testRequest_1) + return maybeWait(d) + def _testRequest_1(self, res): + c = interfaces.IControl(self.master) req = base.BuildRequest("I was bored", SourceStamp()) builder_control = c.getBuilder("force") d = defer.Deferred() req.subscribe(d.callback) builder_control.requestBuild(req) - d.addCallback(self._testForce_1) - return maybeWait(d) + d.addCallback(self._testForce_2) + # we use the same check-the-results code as testForce + return d Index: test_status.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_status.py,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- test_status.py 9 Aug 2005 00:43:35 -0000 1.23 +++ test_status.py 14 Oct 2005 19:47:58 -0000 1.24 @@ -4,7 +4,6 @@ from twisted.internet import defer, reactor from twisted.trial import unittest -dr = unittest.deferredResult from buildbot import interfaces from buildbot.sourcestamp import SourceStamp @@ -124,6 +123,11 @@ def setUp(self): self.builder = MyBuilder("builder1") + def stall(self, res, timeout): + d = defer.Deferred() + reactor.callLater(timeout, d.callback, res) + return d + def makeBuild(self, number, results): return MyBuild(self.builder, number, results) @@ -339,16 +343,15 @@ print "sending mail to", dest d = mailer.buildFinished("builder1", b1, b1.results) - dr(d) - # the mail has been sent, but the SMTP connection is still up - # (because smtp.sendmail relies upon the server to hang up). Spin for - # a moment to avoid the "unclean reactor" warning that Trial gives us - # if we finish before the socket is disconnected. Really, sendmail() - # ought to hang up the connection once it is finished: otherwise a - # malicious SMTP server could make us consume lots of memory. - d = defer.Deferred() - reactor.callLater(0.1, d.callback, None) - dr(d) + # When this fires, the mail has been sent, but the SMTP connection is + # still up (because smtp.sendmail relies upon the server to hang up). + # Spin for a moment to avoid the "unclean reactor" warning that Trial + # gives us if we finish before the socket is disconnected. Really, + # sendmail() ought to hang up the connection once it is finished: + # otherwise a malicious SMTP server could make us consume lots of + # memory. + d.addCallback(self.stall, 0.1) + return maybeWait(d) if not mail: Mail.skip = "the Twisted Mail package is not installed" From warner at users.sourceforge.net Fri Oct 14 19:48:00 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 19:48:00 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.509,1.510 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1561 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-328 Creator: Brian Warner remove almost all remaining uses of deferredResult from unit tests * buildbot/test/test_changes.py: remove use of deferredResult * buildbot/test/test_config.py: same * buildbot/test/test_control.py: same * buildbot/test/test_status.py: same * buildbot/test/test_vc.py: this is the only remaining use, since it gets used at module level. This needs to be replaced by some sort of class-level run-once routine. Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.509 retrieving revision 1.510 diff -u -d -r1.509 -r1.510 --- ChangeLog 14 Oct 2005 19:42:40 -0000 1.509 +++ ChangeLog 14 Oct 2005 19:47:58 -0000 1.510 @@ -1,5 +1,13 @@ 2005-10-13 Brian Warner + * buildbot/test/test_changes.py: remove use of deferredResult + * buildbot/test/test_config.py: same + * buildbot/test/test_control.py: same + * buildbot/test/test_status.py: same + * buildbot/test/test_vc.py: this is the only remaining use, since + it gets used at module level. This needs to be replaced by some + sort of class-level run-once routine. + * buildbot/status/words.py (IrcStatusBot.command_WATCH): fix typo * lots: implement multiple slaves per Builder, which means multiple From warner at users.sourceforge.net Fri Oct 14 20:07:59 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 20:07:59 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_config.py,1.27,1.28 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6686/buildbot/test Modified Files: test_config.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-330 Creator: Brian Warner oops, fix bug resulting from deferredResult changes Index: test_config.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_config.py,v retrieving revision 1.27 retrieving revision 1.28 diff -u -d -r1.27 -r1.28 --- test_config.py 14 Oct 2005 19:47:58 -0000 1.27 +++ test_config.py 14 Oct 2005 20:07:57 -0000 1.28 @@ -460,7 +460,7 @@ master.loadConfig(emptyCfg) self.failUnlessEqual(list(master.change_svc), []) - sourcesCfg = emptyCfg + \ + self.sourcesCfg = emptyCfg + \ """ from buildbot.changes.freshcvs import FreshCVSSource s1 = FreshCVSSource('cvs.example.com', 1000, 'pname', 'spass', @@ -468,7 +468,7 @@ c['sources'] = [s1] """ - d = master.loadConfig(sourcesCfg) + d = master.loadConfig(self.sourcesCfg) d.addCallback(self._testSources_1) return maybeWait(d) @@ -482,7 +482,7 @@ self.failUnless(s1.parent) # verify that unchanged sources are not interrupted - d = self.buildmaster.loadConfig(sourcesCfg) + d = self.buildmaster.loadConfig(self.sourcesCfg) d.addCallback(self._testSources_2) return d From warner at users.sourceforge.net Fri Oct 14 20:07:59 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 14 Oct 2005 20:07:59 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.510,1.511 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6686 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-330 Creator: Brian Warner oops, fix bug resulting from deferredResult changes Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.510 retrieving revision 1.511 diff -u -d -r1.510 -r1.511 --- ChangeLog 14 Oct 2005 19:47:58 -0000 1.510 +++ ChangeLog 14 Oct 2005 20:07:57 -0000 1.511 @@ -1,3 +1,8 @@ +2005-10-14 Brian Warner + + * buildbot/test/test_config.py (ConfigTest._testSources_1): oops, + fix bug resulting from deferredResult changes + 2005-10-13 Brian Warner * buildbot/test/test_changes.py: remove use of deferredResult From warner at users.sourceforge.net Sat Oct 15 19:19:20 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 15 Oct 2005 19:19:20 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_config.py,1.28,1.29 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29889/buildbot/test Modified Files: test_config.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-332 Creator: Brian Warner fix the testSources bug, for real this time Index: test_config.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_config.py,v retrieving revision 1.28 retrieving revision 1.29 diff -u -d -r1.28 -r1.29 --- test_config.py 14 Oct 2005 20:07:57 -0000 1.28 +++ test_config.py 15 Oct 2005 19:19:18 -0000 1.29 @@ -483,10 +483,10 @@ # verify that unchanged sources are not interrupted d = self.buildmaster.loadConfig(self.sourcesCfg) - d.addCallback(self._testSources_2) + d.addCallback(self._testSources_2, s1) return d - def _testSources_2(self, res): + def _testSources_2(self, res, s1): self.failUnlessEqual(len(list(self.buildmaster.change_svc)), 1) s2 = list(self.buildmaster.change_svc)[0] self.failUnlessIdentical(s1, s2) From warner at users.sourceforge.net Sun Oct 16 06:25:36 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Oct 2005 06:25:36 +0000 Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.33,1.34 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3074/buildbot Modified Files: interfaces.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-334 Creator: Brian Warner add getURLForThing status method * buildbot/status/builder.py (Status.getURLForThing): add method to provide a URL for arbitrary IStatusFoo objects. The idea is to use this in email/IRC status clients to make them more useful, by providing the end user with hints on where to learn more about the object being reported on. * buildbot/test/test_web.py (GetURL): tests for it Index: interfaces.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v retrieving revision 1.33 retrieving revision 1.34 diff -u -d -r1.33 -r1.34 --- interfaces.py 14 Oct 2005 19:42:39 -0000 1.33 +++ interfaces.py 16 Oct 2005 06:25:34 -0000 1.34 @@ -96,6 +96,11 @@ def getBuildbotURL(): """Return the URL of the top-most Buildbot status page, or None if this Buildbot does not provide a web status page.""" + def getURLFor(thing): + """Return the URL of a page which provides information on 'thing', + which should be an object that implements one of the status + interfaces defined in L{buildbot.interfaces}. Returns None if no + suitable page is available (or if no Waterfall is running).""" def getSchedulers(): """Return a list of ISchedulerStatus objects for all @@ -207,6 +212,7 @@ def getPendingBuildsets(): """Return an IBuildSet for all BuildSets that are pending. These BuildSets are waiting for their tree-stable-timers to expire.""" + # TODO: this is not implemented anywhere class IBuilderStatus(Interface): From warner at users.sourceforge.net Sun Oct 16 06:25:36 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Oct 2005 06:25:36 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.511,1.512 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3074 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-334 Creator: Brian Warner add getURLForThing status method * buildbot/status/builder.py (Status.getURLForThing): add method to provide a URL for arbitrary IStatusFoo objects. The idea is to use this in email/IRC status clients to make them more useful, by providing the end user with hints on where to learn more about the object being reported on. * buildbot/test/test_web.py (GetURL): tests for it Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.511 retrieving revision 1.512 diff -u -d -r1.511 -r1.512 --- ChangeLog 14 Oct 2005 20:07:57 -0000 1.511 +++ ChangeLog 16 Oct 2005 06:25:34 -0000 1.512 @@ -1,3 +1,12 @@ +2005-10-15 Brian Warner + + * buildbot/status/builder.py (Status.getURLForThing): add method + to provide a URL for arbitrary IStatusFoo objects. The idea is to + use this in email/IRC status clients to make them more useful, by + providing the end user with hints on where to learn more about the + object being reported on. + * buildbot/test/test_web.py (GetURL): tests for it + 2005-10-14 Brian Warner * buildbot/test/test_config.py (ConfigTest._testSources_1): oops, From warner at users.sourceforge.net Sun Oct 16 06:25:37 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Oct 2005 06:25:37 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.68,1.69 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3074/buildbot/status Modified Files: builder.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-334 Creator: Brian Warner add getURLForThing status method * buildbot/status/builder.py (Status.getURLForThing): add method to provide a URL for arbitrary IStatusFoo objects. The idea is to use this in email/IRC status clients to make them more useful, by providing the end user with hints on where to learn more about the object being reported on. * buildbot/test/test_web.py (GetURL): tests for it Index: builder.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v retrieving revision 1.68 retrieving revision 1.69 diff -u -d -r1.68 -r1.69 --- builder.py 14 Oct 2005 19:42:40 -0000 1.68 +++ builder.py 16 Oct 2005 06:25:34 -0000 1.69 @@ -15,7 +15,7 @@ # sibling imports from buildbot import interfaces, util, sourcestamp -from buildbot.twcompat import implements +from buildbot.twcompat import implements, providedBy SUCCESS, WARNINGS, FAILURE, SKIPPED, EXCEPTION = range(5) Results = ["success", "warnings", "failure", "skipped", "exception"] @@ -1708,6 +1708,63 @@ def getBuildbotURL(self): return self.botmaster.parent.buildbotURL + def getURLForThing(self, thing): + prefix = self.getBuildbotURL() + if not prefix: + return None + if providedBy(thing, interfaces.IStatus): + return prefix + if providedBy(thing, interfaces.ISchedulerStatus): + pass + if providedBy(thing, interfaces.IBuilderStatus): + builder = thing + return prefix + builder.getName() + if providedBy(thing, interfaces.IBuildStatus): + build = thing + builder = build.getBuilder() + return "%s%s/builds/%d" % (prefix, + builder.getName(), + build.getNumber()) + if providedBy(thing, interfaces.IBuildStepStatus): + step = thing + build = step.getBuild() + builder = build.getBuilder() + return "%s%s/builds/%d/%s" % (prefix, + builder.getName(), + build.getNumber(), + step.getName()) + # IBuildSetStatus + # IBuildRequestStatus + # ISlaveStatus + + # IStatusEvent + if providedBy(thing, interfaces.IStatusEvent): + from buildbot.changes import changes + # TODO: this is goofy, create IChange or something + if isinstance(thing, changes.Change): + change = thing + return "%schanges/%d" % (prefix, change.number) + + if providedBy(thing, interfaces.IStatusLog): + log = thing + step = log.getStep() + build = step.getBuild() + builder = build.getBuilder() + + logs = step.getLogs() + for i in range(len(logs)): + if log is logs[i]: + lognum = i + break + else: + return None + return "%s%s/builds/%d/%s/%d" % (prefix, + builder.getName(), + build.getNumber(), + step.getName(), + lognum) + + def getSchedulers(self): return self.botmaster.parent.schedulers From warner at users.sourceforge.net Sun Oct 16 06:25:36 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 16 Oct 2005 06:25:36 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_web.py,1.21,1.22 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3074/buildbot/test Modified Files: test_web.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-334 Creator: Brian Warner add getURLForThing status method * buildbot/status/builder.py (Status.getURLForThing): add method to provide a URL for arbitrary IStatusFoo objects. The idea is to use this in email/IRC status clients to make them more useful, by providing the end user with hints on where to learn more about the object being reported on. * buildbot/test/test_web.py (GetURL): tests for it Index: test_web.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_web.py,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- test_web.py 9 Aug 2005 00:43:35 -0000 1.21 +++ test_web.py 16 Oct 2005 06:25:34 -0000 1.22 @@ -5,6 +5,7 @@ #log.startLogging(sys.stderr) from twisted.trial import unittest +from buildbot.test.runutils import RunMixin from twisted.internet import reactor, defer, protocol from twisted.internet.interfaces import IReactorUNIX @@ -99,6 +100,14 @@ self.p.factory = self return self.p +def stopHTTPLog(): + # grr. + try: + from twisted.web import http # Twisted-2.0 + except ImportError: + from twisted.protocols import http # Twisted-1.3 + http._logDateTimeStop() + class BaseWeb: master = None @@ -106,12 +115,7 @@ self.failUnless(string.find(substr) != -1) def tearDown(self): - # grr. - try: - from twisted.web import http # Twisted-2.0 - except ImportError: - from twisted.protocols import http # Twisted-1.3 - http._logDateTimeStop() + stopHTTPLog() if self.master: d = self.master.stopService() return maybeWait(d) @@ -231,7 +235,119 @@ self.failUnlessIn("
  • Syncmail mailing list in maildir " + "my-maildir
  • ", changes) -class Logfile(BaseWeb, unittest.TestCase): + +geturl_config = """ +from buildbot.status import html +from buildbot.changes import mail +from buildbot.process import step, factory +from buildbot.scheduler import Scheduler +from buildbot.changes.base import ChangeSource +s = factory.s + +class DiscardScheduler(Scheduler): + def addChange(self, change): + pass +class DummyChangeSource(ChangeSource): + pass + +BuildmasterConfig = c = {} +c['bots'] = [('bot1', 'sekrit'), ('bot2', 'sekrit')] +c['sources'] = [DummyChangeSource()] +c['schedulers'] = [DiscardScheduler('discard', None, 60, ['b1'])] +c['slavePortnum'] = 0 +c['status'] = [html.Waterfall(http_port=0)] + +f = factory.BuildFactory([s(step.RemoteDummy, timeout=1)]) + +c['builders'] = [ + {'name': 'b1', 'slavenames': ['bot1','bot2'], + 'builddir': 'b1', 'factory': f}, + ] +c['buildbotURL'] = 'http://dummy.example.org:8010/' + +""" + +class GetURL(RunMixin, unittest.TestCase): + + def setUp(self): + RunMixin.setUp(self) + self.master.loadConfig(geturl_config) + self.master.startService() + d = self.connectSlave(["b1"]) + return maybeWait(d) + + def tearDown(self): + stopHTTPLog() + return RunMixin.tearDown(self) + + def doBuild(self, buildername): + br = base.BuildRequest("forced", sourcestamp.SourceStamp()) + d = self.control.getBuilder(buildername).requestBuild(br) + return d + + def assertNoURL(self, target): + self.failUnlessIdentical(self.status.getURLForThing(target), None) + + def assertURLEqual(self, target, expected): + got = self.status.getURLForThing(target) + full_expected = "http://dummy.example.org:8010/" + expected + self.failUnlessEqual(got, full_expected) + + def testMissingBase(self): + noweb_config1 = geturl_config + "del c['buildbotURL']\n" + d = self.master.loadConfig(noweb_config1) + d.addCallback(self._testMissingBase_1) + return maybeWait(d) + def _testMissingBase_1(self, res): + s = self.status + self.assertNoURL(s) + builder = s.getBuilder("b1") + self.assertNoURL(builder) + + def testBase(self): + s = self.status + self.assertURLEqual(s, "") + builder = s.getBuilder("b1") + self.assertURLEqual(builder, "b1") + + def testBrokenStuff(self): + s = self.status + self.assertURLEqual(s.getSchedulers()[0], "schedulers/0") + self.assertURLEqual(s.getSlave("bot1"), "slaves/bot1") + # we didn't put a Change into the actual Build before, so this fails + #self.assertURLEqual(build.getChanges()[0], "changes/1") + testBrokenStuff.todo = "not implemented yet" + + def testChange(self): + s = self.status + c = Change("user", ["foo.c"], "comments") + self.master.change_svc.addChange(c) + # TODO: something more like s.getChanges(), requires IChange and + # an accessor in IStatus. The HTML page exists already, though + self.assertURLEqual(c, "changes/1") + + def testBuild(self): + # first we do some stuff so we'll have things to look at. + s = self.status + d = self.doBuild("b1") + # maybe check IBuildSetStatus here? + d.addCallback(self._testBuild_1) + return maybeWait(d) + + def _testBuild_1(self, res): + s = self.status + builder = s.getBuilder("b1") + build = builder.getLastFinishedBuild() + self.assertURLEqual(build, "b1/builds/0") + # no page for builder.getEvent(-1) + step = build.getSteps()[0] + self.assertURLEqual(step, "b1/builds/0/remote dummy") # really? + # maybe page for build.getTestResults? + self.assertURLEqual(step.getLogs()[0], "b1/builds/0/remote dummy/0") + + + +class Logfile(BaseWeb, RunMixin, unittest.TestCase): def setUp(self): config = """ from buildbot.status import html From warner at users.sourceforge.net Tue Oct 18 06:29:58 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 18 Oct 2005 06:29:58 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.512,1.513 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12309 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-336 Creator: Brian Warner stop using deprecated Trial arguments, at least for TwistedTrial * buildbot/process/process_twisted.py (TwistedTrial): update Trial arguments to accomodate Twisted >=2.1.0 . I will have to figure out what to do about other projects: the correct options for recent Twisteds will not work for older ones. Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.512 retrieving revision 1.513 diff -u -d -r1.512 -r1.513 --- ChangeLog 16 Oct 2005 06:25:34 -0000 1.512 +++ ChangeLog 18 Oct 2005 06:29:56 -0000 1.513 @@ -1,3 +1,10 @@ +2005-10-17 Brian Warner + + * buildbot/process/process_twisted.py (TwistedTrial): update Trial + arguments to accomodate Twisted >=2.1.0 . I will have to figure + out what to do about other projects: the correct options for + recent Twisteds will not work for older ones. + 2005-10-15 Brian Warner * buildbot/status/builder.py (Status.getURLForThing): add method From warner at users.sourceforge.net Tue Oct 18 06:29:58 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 18 Oct 2005 06:29:58 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process process_twisted.py,1.38,1.39 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12309/buildbot/process Modified Files: process_twisted.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-336 Creator: Brian Warner stop using deprecated Trial arguments, at least for TwistedTrial * buildbot/process/process_twisted.py (TwistedTrial): update Trial arguments to accomodate Twisted >=2.1.0 . I will have to figure out what to do about other projects: the correct options for recent Twisteds will not work for older ones. Index: process_twisted.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/process_twisted.py,v retrieving revision 1.38 retrieving revision 1.39 diff -u -d -r1.38 -r1.39 --- process_twisted.py 17 Jul 2005 23:28:33 -0000 1.38 +++ process_twisted.py 18 Oct 2005 06:29:56 -0000 1.39 @@ -19,7 +19,10 @@ class TwistedTrial(Trial): tests = "twisted" - recurse = True + # the Trial in Twisted >=2.1.0 has --recurse on by default, and -to + # turned into --reporter=bwverbose . + recurse = False + trialMode = "--reporter=bwverbose" testpath = None trial = "./bin/trial" From warner at users.sourceforge.net Wed Oct 19 05:59:36 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Wed, 19 Oct 2005 05:59:36 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.69,1.70 words.py,1.41,1.42 mail.py,1.19,1.20 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11342/buildbot/status Modified Files: html.py words.py mail.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-338 Creator: Brian Warner use getURLForThing to add per-Build URLs to status messages * buildbot/status/html.py: provide 'status' argument to most StatusResourceFOO objects (StatusResourceBuild.body): href-ify the Builder name, add "Steps and Logfiles" section to make the Build page into a more-or-less comprehensive source of status information about the build * buildbot/status/mail.py (MailNotifier): include the Build's URL * buildbot/status/words.py (IrcStatusBot.buildFinished): same Index: html.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v retrieving revision 1.69 retrieving revision 1.70 diff -u -d -r1.69 -r1.70 --- html.py 14 Oct 2005 19:42:40 -0000 1.69 +++ html.py 19 Oct 2005 05:59:34 -0000 1.70 @@ -176,8 +176,9 @@ class StatusResourceBuildStep(HtmlResource): title = "Build Step" - def __init__(self, step): + def __init__(self, status, step): HtmlResource.__init__(self) + self.status = status self.step = step def body(self, request): @@ -231,8 +232,9 @@ class StatusResourceTestResult(HtmlResource): title = "Test Logs" - def __init__(self, name, result): + def __init__(self, status, name, result): HtmlResource.__init__(self) + self.status = status self.name = name self.result = result @@ -253,8 +255,9 @@ class StatusResourceTestResults(HtmlResource): title = "Test Results" - def __init__(self, results): + def __init__(self, status, results): HtmlResource.__init__(self) + self.status = status self.results = results def body(self, request): @@ -281,7 +284,7 @@ try: name = tuple(path.split(".")) result = self.results[name] - return StatusResourceTestResult(name, result) + return StatusResourceTestResult(self.status, name, result) except KeyError: return NoResource("No such test name '%s'" % path) @@ -290,17 +293,19 @@ class StatusResourceBuild(HtmlResource): title = "Build" - def __init__(self, build, control): + def __init__(self, status, build, control): HtmlResource.__init__(self) + self.status = status self.build = build self.control = control def body(self, request): b = self.build # the color in the following line gives python-mode trouble - data = ("

    Build %s:#%d

    \n" + data = ("

    Build %s:#%d

    \n" "

    Reason:

    \n%s\n" - % (b.getBuilder().getName(), b.getNumber(), + % (self.status.getURLForThing(b.getBuilder()), + urllib.quote(b.getBuilder().getName()), b.getNumber(), html.escape(b.getReason()))) if b.isFinished(): data += "

    Buildslave: %s

    \n" % html.escape(b.getSlavename()) @@ -325,6 +330,22 @@ """ + data += "

    Steps and Logfiles:

    \n" + if b.getLogs(): + data += "
      \n" + for s in b.getSteps(): + data += "
    1. %s [%s]\n" % (s.getName(), + " ".join(s.getText())) + if s.getLogs(): + data += "
        \n" + for logfile in s.getLogs(): + data += ("
      1. %s
      2. \n" % + (self.status.getURLForThing(logfile), + logfile.getName())) + data += "
      \n" + data += "
    2. \n" + data += "
    \n" + data += ("

    Blamelist:

    \n" "
      \n") for who in b.getResponsibleUsers(): @@ -362,21 +383,23 @@ def getChild(self, path, request): if path == "tests": # TODO: this will collide with a step named 'tests' - return StatusResourceTestResults(self.build.getTestResults()) + return StatusResourceTestResults(self.status, + self.build.getTestResults()) if path == "stop": return self.stop(request) stepname = path steps = self.build.getSteps() for s in steps: if s.getName() == stepname: - return StatusResourceBuildStep(s) + return StatusResourceBuildStep(self.status, s) return NoResource("No such BuildStep '%s'" % stepname) # $builder class StatusResourceBuilder(HtmlResource): - def __init__(self, builder, control): + def __init__(self, status, builder, control): HtmlResource.__init__(self) + self.status = status self.title = builder.getName() + " Builder" self.builder = builder self.control = control @@ -498,15 +521,16 @@ control = None if self.control: control = self.control.getBuild(num) - return StatusResourceBuild(build, control) + return StatusResourceBuild(self.status, build, control) else: return NoResource("No such build '%d'" % num) return NoResource("really weird URL %s" % path) # $changes/NN class StatusResourceChanges(HtmlResource): - def __init__(self, changemaster): + def __init__(self, status, changemaster): HtmlResource.__init__(self) + self.status = status self.changemaster = changemaster def body(self, request): data = "" @@ -1411,7 +1435,7 @@ if path == "buildbot.css" and self.css: return static.File(self.css) if path == "changes": - return StatusResourceChanges(self.changemaster) + return StatusResourceChanges(self.status, self.changemaster) if path == "favicon.ico": if self.favicon: return static.File(self.favicon) @@ -1422,7 +1446,7 @@ control = None if self.control: control = self.control.getBuilder(path) - return StatusResourceBuilder(builder, control) + return StatusResourceBuilder(self.status, builder, control) return NoResource("No such Builder '%s'" % path) Index: mail.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/mail.py,v retrieving revision 1.19 retrieving revision 1.20 diff -u -d -r1.19 -r1.20 --- mail.py 31 Aug 2005 08:04:36 -0000 1.19 +++ mail.py 19 Oct 2005 05:59:34 -0000 1.20 @@ -230,6 +230,9 @@ text += "The Buildbot has detected a failed build of %s.\n" % name else: text += "The Buildbot has detected a new failure of %s.\n" % name + buildurl = self.status.getURLForThing(build) + if buildurl: + text += "Full details are available at: %s\n" % buildurl text += "\n" url = self.status.getBuildbotURL() Index: words.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/words.py,v retrieving revision 1.41 retrieving revision 1.42 diff -u -d -r1.41 -r1.42 --- words.py 14 Oct 2005 19:42:40 -0000 1.41 +++ words.py 19 Oct 2005 05:59:34 -0000 1.42 @@ -280,6 +280,9 @@ results.get(b.getResults(), "??")) r += " [%s]" % " ".join(b.getText()) self.reply(reply, r) + buildurl = self.status.getURLForThing(b) + if buildurl: + self.reply(reply, "Build details are at %s" % buildurl) def command_FORCE(self, user, reply, args): args = args.split(None, 2) From warner at users.sourceforge.net Wed Oct 19 05:59:36 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Wed, 19 Oct 2005 05:59:36 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.513,1.514 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11342 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-338 Creator: Brian Warner use getURLForThing to add per-Build URLs to status messages * buildbot/status/html.py: provide 'status' argument to most StatusResourceFOO objects (StatusResourceBuild.body): href-ify the Builder name, add "Steps and Logfiles" section to make the Build page into a more-or-less comprehensive source of status information about the build * buildbot/status/mail.py (MailNotifier): include the Build's URL * buildbot/status/words.py (IrcStatusBot.buildFinished): same Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.513 retrieving revision 1.514 diff -u -d -r1.513 -r1.514 --- ChangeLog 18 Oct 2005 06:29:56 -0000 1.513 +++ ChangeLog 19 Oct 2005 05:59:34 -0000 1.514 @@ -1,3 +1,14 @@ +2005-10-18 Brian Warner + + * buildbot/status/html.py: provide 'status' argument to most + StatusResourceFOO objects + (StatusResourceBuild.body): href-ify the Builder name, add "Steps + and Logfiles" section to make the Build page into a more-or-less + comprehensive source of status information about the build + + * buildbot/status/mail.py (MailNotifier): include the Build's URL + * buildbot/status/words.py (IrcStatusBot.buildFinished): same + 2005-10-17 Brian Warner * buildbot/process/process_twisted.py (TwistedTrial): update Trial From warner at users.sourceforge.net Wed Oct 19 06:26:13 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Wed, 19 Oct 2005 06:26:13 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_status.py,1.24,1.25 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15900/buildbot/test Modified Files: test_status.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-340 Creator: Brian Warner oops, fix test_status with a dummy getURLForThing Index: test_status.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_status.py,v retrieving revision 1.24 retrieving revision 1.25 diff -u -d -r1.24 -r1.25 --- test_status.py 14 Oct 2005 19:47:58 -0000 1.24 +++ test_status.py 19 Oct 2005 06:26:10 -0000 1.25 @@ -137,6 +137,9 @@ def getBuildbotURL(self): return "BUILDBOT_URL" + def getURLForThing(self, thing): + return None + def testBuild1(self): mailer = MyMailer(fromaddr="buildbot at example.com", extraRecipients=["recip at example.com", From warner at users.sourceforge.net Wed Oct 19 07:06:03 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Wed, 19 Oct 2005 07:06:03 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.514,1.515 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22065 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-342 Creator: Brian Warner re-sync twisted_master.cfg with reality, pre-Schedulers Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.514 retrieving revision 1.515 diff -u -d -r1.514 -r1.515 --- ChangeLog 19 Oct 2005 05:59:34 -0000 1.514 +++ ChangeLog 19 Oct 2005 07:06:01 -0000 1.515 @@ -1,3 +1,8 @@ +2005-10-19 Brian Warner + + * docs/examples/twisted_master.cfg: re-sync with reality: bring + back python2.2 tests, turn off OS-X threadedselect-reactor tests + 2005-10-18 Brian Warner * buildbot/status/html.py: provide 'status' argument to most From warner at users.sourceforge.net Wed Oct 19 07:06:04 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Wed, 19 Oct 2005 07:06:04 +0000 Subject: [Buildbot-commits] buildbot/docs/examples twisted_master.cfg,1.29,1.30 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/examples In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22065/docs/examples Modified Files: twisted_master.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-342 Creator: Brian Warner re-sync twisted_master.cfg with reality, pre-Schedulers Index: twisted_master.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/examples/twisted_master.cfg,v retrieving revision 1.29 retrieving revision 1.30 diff -u -d -r1.29 -r1.30 --- twisted_master.cfg 3 Sep 2005 19:50:00 -0000 1.29 +++ twisted_master.cfg 19 Oct 2005 07:06:01 -0000 1.30 @@ -80,10 +80,19 @@ 'slavename': "bot1", 'builddir': "quick", 'factory': QuickTwistedBuildFactory(svnurl, - python=["python2.3"]), + python=["python2.2", "python2.3"]), } builders.append(b1) +b22 = {'name': "full-2.2", + 'slavename': "bot-exarkun", + 'builddir': "full2.2", + 'factory': FullTwistedBuildFactory(svnurl, + python="python2.2", + processDocs=0), + } +builders.append(b22) + b23compile_opts = [ "-Wignore::PendingDeprecationWarning:distutils.command.build_py", "-Wignore::PendingDeprecationWarning:distutils.command.build_ext", @@ -140,15 +149,17 @@ 'builddir': "OSX-full2.4", 'factory': TwistedReactorsBuildFactory(svnurl, python="python2.4", - reactors=["default", #"cf", - "threadedselect"], + reactors=["default", + # "cf", + # "threadedselect", + ], ), } builders.append(b23osx) -b23w32 = {'name': "win32", +b22w32 = {'name': "win32", 'slavename': "bot-w32", - 'builddir': "W32-full2.3", + 'builddir': "W32-full2.2", 'factory': TwistedReactorsBuildFactory(svnurl, python="python", compileOpts2=["-c","mingw32"], @@ -157,11 +168,11 @@ "win32", ]), } -builders.append(b23w32) +builders.append(b22w32) b23bsd = {'name': "freebsd", 'slavename': "bot-suszko", - 'builddir': "bsd-full2.3", + 'builddir': "bsd-full2.2", 'factory': TwistedReactorsBuildFactory(svnurl, python="python2.3", reactors=["default", @@ -196,7 +207,7 @@ channels=["twisted"])) c['debugPassword'] = private.debugPassword -#c['interlocks'] = [("do-deb", ["full-2.3"], ["debuild"])] +#c['interlocks'] = [("do-deb", ["full-2.2"], ["debuild"])] if hasattr(private, "manhole"): c['manhole'] = master.Manhole(*private.manhole) c['status'].append(client.PBListener(9936)) From warner at users.sourceforge.net Thu Oct 20 22:25:31 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:25:31 +0000 Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.34,1.35 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9509/buildbot Modified Files: interfaces.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-344 Creator: Brian Warner clean up use of 'force build' methods * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new method specifically for use by HTML "force build" button and the IRC "force" command. Raises an immediate error if there are no slaves available. (IBuilderControl.requestBuild): make this just submit a build, not try to check for existing slaves or set up any when-finished Deferreds or anything. * buildbot/process/builder.py (BuilderControl): same * buildbot/status/html.py (StatusResourceBuilder.force): same * buildbot/status/words.py (IrcStatusBot.command_FORCE): same * buildbot/test/test_slaves.py: same * buildbot/test/test_web.py: same Index: interfaces.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v retrieving revision 1.34 retrieving revision 1.35 diff -u -d -r1.34 -r1.35 --- interfaces.py 16 Oct 2005 06:25:34 -0000 1.34 +++ interfaces.py 20 Oct 2005 22:25:29 -0000 1.35 @@ -819,9 +819,14 @@ def requestBuild(request): """Queue a L{buildbot.process.base.BuildRequest} object for later - building. This returns a Deferred that fires (with an L{IBuildStatus} - instance) when the BuildRequest finishes, just as if you did - req.waitUntilFinished.""" + building.""" + + def requestBuildSoon(request): + """Submit a BuildRequest like requestBuild, but raise a + L{buildbot.interfaces.NoSlaveError} if no slaves are currently + available, so it cannot be used to queue a BuildRequest in the hopes + that a slave will eventually connect. This method is appropriate for + use by things like the web-page 'Force Build' button.""" def getPendingBuilds(): """Return a list of L{IBuildRequestControl} objects for this Builder. From warner at users.sourceforge.net Thu Oct 20 22:25:31 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:25:31 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.515,1.516 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9509 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-344 Creator: Brian Warner clean up use of 'force build' methods * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new method specifically for use by HTML "force build" button and the IRC "force" command. Raises an immediate error if there are no slaves available. (IBuilderControl.requestBuild): make this just submit a build, not try to check for existing slaves or set up any when-finished Deferreds or anything. * buildbot/process/builder.py (BuilderControl): same * buildbot/status/html.py (StatusResourceBuilder.force): same * buildbot/status/words.py (IrcStatusBot.command_FORCE): same * buildbot/test/test_slaves.py: same * buildbot/test/test_web.py: same Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.515 retrieving revision 1.516 diff -u -d -r1.515 -r1.516 --- ChangeLog 19 Oct 2005 07:06:01 -0000 1.515 +++ ChangeLog 20 Oct 2005 22:25:29 -0000 1.516 @@ -1,3 +1,18 @@ +2005-10-20 Brian Warner + + * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new + method specifically for use by HTML "force build" button and the + IRC "force" command. Raises an immediate error if there are no + slaves available. + (IBuilderControl.requestBuild): make this just submit a build, not + try to check for existing slaves or set up any when-finished + Deferreds or anything. + * buildbot/process/builder.py (BuilderControl): same + * buildbot/status/html.py (StatusResourceBuilder.force): same + * buildbot/status/words.py (IrcStatusBot.command_FORCE): same + * buildbot/test/test_slaves.py: same + * buildbot/test/test_web.py: same + 2005-10-19 Brian Warner * docs/examples/twisted_master.cfg: re-sync with reality: bring From warner at users.sourceforge.net Thu Oct 20 22:25:31 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:25:31 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.70,1.71 words.py,1.42,1.43 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9509/buildbot/status Modified Files: html.py words.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-344 Creator: Brian Warner clean up use of 'force build' methods * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new method specifically for use by HTML "force build" button and the IRC "force" command. Raises an immediate error if there are no slaves available. (IBuilderControl.requestBuild): make this just submit a build, not try to check for existing slaves or set up any when-finished Deferreds or anything. * buildbot/process/builder.py (BuilderControl): same * buildbot/status/html.py (StatusResourceBuilder.force): same * buildbot/status/words.py (IrcStatusBot.command_FORCE): same * buildbot/test/test_slaves.py: same * buildbot/test/test_web.py: same Index: html.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v retrieving revision 1.70 retrieving revision 1.71 diff -u -d -r1.70 -r1.71 --- html.py 19 Oct 2005 05:59:34 -0000 1.70 +++ html.py 20 Oct 2005 22:25:29 -0000 1.71 @@ -20,8 +20,10 @@ from buildbot import interfaces, util from buildbot import version +from buildbot.sourcestamp import SourceStamp from buildbot.status import builder, base from buildbot.changes import changes +from buildbot.process.base import BuildRequest class ITopBox(Interface): """I represent a box in the top row of the waterfall display: the one @@ -472,9 +474,12 @@ # TODO: if we can authenticate that a particular User pushed the # button, use their name instead of None, so they'll be informed of # the results. + branch = None + s = SourceStamp(branch=branch) + req = BuildRequest(r, s, self.builder.getName()) try: - self.control.forceBuild(None, r) - except (interfaces.NoSlaveError, interfaces.BuilderInUseError): + self.control.requestBuildSoon(req) + except interfaces.NoSlaveError: # TODO: tell the web user that their request could not be # honored pass Index: words.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/words.py,v retrieving revision 1.42 retrieving revision 1.43 diff -u -d -r1.42 -r1.43 --- words.py 19 Oct 2005 05:59:34 -0000 1.42 +++ words.py 20 Oct 2005 22:25:29 -0000 1.43 @@ -17,6 +17,8 @@ from buildbot import interfaces, util from buildbot import version +from buildbot.sourcestamp import SourceStamp +from buildbot.process.base import BuildRequest from buildbot.status import base from buildbot.status.builder import SUCCESS, WARNINGS, FAILURE, EXCEPTION @@ -299,25 +301,20 @@ # TODO: or, monitor this build and announce the results through the # 'reply' argument. r = "forced: by IRC user <%s>: %s" % (user, reason) + # TODO: maybe give certain users the ability to request builds of + # certain branches + branch = None + s = SourceStamp(branch=branch) + req = BuildRequest(r, s, which) try: - # TODO: replace this with bc.requestBuild, and maybe give certain - # users the ability to request builds of certain branches - d = bc.forceBuild(who, r) + bc.requestBuildSoon(req) except interfaces.NoSlaveError: self.reply(reply, - "sorry, I can't force a build: the slave is offline") - return - except interfaces.BuilderInUseError: - self.reply(reply, - "sorry, I can't force a build: the slave is in use") - return - if not d: - self.reply(reply, "sorry, I can't force a build: I must have " - "left the builder in my other pants") + "sorry, I can't force a build: all slaves are offline") return + ireq = IrcBuildRequest(self, reply) + req.subscribe(ireq.started) - req = IrcBuildRequest(self, reply) - d.addCallback(req.started) command_FORCE.usage = "force build - Force a build" From warner at users.sourceforge.net Thu Oct 20 22:25:31 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:25:31 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_web.py,1.22,1.23 test_slaves.py,1.2,1.3 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9509/buildbot/test Modified Files: test_web.py test_slaves.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-344 Creator: Brian Warner clean up use of 'force build' methods * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new method specifically for use by HTML "force build" button and the IRC "force" command. Raises an immediate error if there are no slaves available. (IBuilderControl.requestBuild): make this just submit a build, not try to check for existing slaves or set up any when-finished Deferreds or anything. * buildbot/process/builder.py (BuilderControl): same * buildbot/status/html.py (StatusResourceBuilder.force): same * buildbot/status/words.py (IrcStatusBot.command_FORCE): same * buildbot/test/test_slaves.py: same * buildbot/test/test_web.py: same Index: test_slaves.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slaves.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- test_slaves.py 14 Oct 2005 19:42:39 -0000 1.2 +++ test_slaves.py 20 Oct 2005 22:25:29 -0000 1.3 @@ -40,7 +40,8 @@ def doBuild(self, buildername): br = BuildRequest("forced", SourceStamp()) - d = self.control.getBuilder(buildername).requestBuild(br) + d = br.waitUntilFinished() + self.control.getBuilder(buildername).requestBuild(br) return d def testSequence(self): @@ -138,7 +139,8 @@ timers = [] self.slaves['bot1'].debugOpts["stallPings"] = (10, timers) br = BuildRequest("forced", SourceStamp()) - d1 = self.control.getBuilder("b1").requestBuild(br) + d1 = br.waitUntilFinished() + self.control.getBuilder("b1").requestBuild(br) s1 = br.status # this is a BuildRequestStatus # give it a chance to start pinging d2 = defer.Deferred() @@ -181,7 +183,8 @@ # step. self.revision += 1 br = BuildRequest(reason, SourceStamp(revision=self.revision)) - d = self.control.getBuilder(buildername).requestBuild(br) + d = br.waitUntilFinished() + self.control.getBuilder(buildername).requestBuild(br) return d def testFirstComeFirstServed(self): Index: test_web.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_web.py,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- test_web.py 16 Oct 2005 06:25:34 -0000 1.22 +++ test_web.py 20 Oct 2005 22:25:29 -0000 1.23 @@ -282,7 +282,8 @@ def doBuild(self, buildername): br = base.BuildRequest("forced", sourcestamp.SourceStamp()) - d = self.control.getBuilder(buildername).requestBuild(br) + d = br.waitUntilFinished() + self.control.getBuilder(buildername).requestBuild(br) return d def assertNoURL(self, target): From warner at users.sourceforge.net Thu Oct 20 22:25:31 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:25:31 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process builder.py,1.32,1.33 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9509/buildbot/process Modified Files: builder.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-344 Creator: Brian Warner clean up use of 'force build' methods * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new method specifically for use by HTML "force build" button and the IRC "force" command. Raises an immediate error if there are no slaves available. (IBuilderControl.requestBuild): make this just submit a build, not try to check for existing slaves or set up any when-finished Deferreds or anything. * buildbot/process/builder.py (BuilderControl): same * buildbot/status/html.py (StatusResourceBuilder.force): same * buildbot/status/words.py (IrcStatusBot.command_FORCE): same * buildbot/test/test_slaves.py: same * buildbot/test/test_web.py: same Index: builder.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/builder.py,v retrieving revision 1.32 retrieving revision 1.33 diff -u -d -r1.32 -r1.33 --- builder.py 14 Oct 2005 19:42:40 -0000 1.32 +++ builder.py 20 Oct 2005 22:25:29 -0000 1.33 @@ -603,7 +603,7 @@ easier. """ - warnings.warn("Please use BuilderControl.requestBuild instead", + warnings.warn("Please use BuilderControl.requestBuildSoon instead", category=DeprecationWarning, stacklevel=1) # see if there is an idle slave, so we can emit an appropriate error @@ -636,13 +636,18 @@ return w.wait() def requestBuild(self, req): - """Submit a BuildRequest to this Builder. Returns a Deferred that - fires when the BuildRequest finishes, the same as doing - req.waitUntilFinished . This Deferred will fire with an - L{buildbot.interfaces.IBuildStatus} instance.""" - d = req.waitUntilFinished() + """Submit a BuildRequest to this Builder.""" self.original.submitBuildRequest(req) - return d + + def requestBuildSoon(self, req): + """Submit a BuildRequest like requestBuild, but raise a + L{buildbot.interfaces.NoSlaveError} if no slaves are currently + available, so it cannot be used to queue a BuildRequest in the hopes + that a slave will eventually connect. This method is appropriate for + use by things like the web-page 'Force Build' button.""" + if not self.original.slaves: + raise interfaces.NoSlaveError + self.requestBuild(req) def getPendingBuilds(self): # return IBuildRequestControl objects From warner at users.sourceforge.net Thu Oct 20 22:32:50 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:32:50 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.516,1.517 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11143 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-346 Creator: Brian Warner update sample master.cfg files to use Schedulers Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.516 retrieving revision 1.517 diff -u -d -r1.516 -r1.517 --- ChangeLog 20 Oct 2005 22:25:29 -0000 1.516 +++ ChangeLog 20 Oct 2005 22:32:48 -0000 1.517 @@ -1,5 +1,11 @@ 2005-10-20 Brian Warner + * buildbot/status/mail.py (MailNotifier.buildMessage): reformat + + * docs/examples/twisted_master.cfg: update to use Schedulers + + * buildbot/scripts/sample.cfg: update with Schedulers + * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new method specifically for use by HTML "force build" button and the IRC "force" command. Raises an immediate error if there are no From warner at users.sourceforge.net Thu Oct 20 22:32:50 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:32:50 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.69,1.70 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11143/buildbot/process Modified Files: step.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-346 Creator: Brian Warner update sample master.cfg files to use Schedulers Index: step.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v retrieving revision 1.69 retrieving revision 1.70 diff -u -d -r1.69 -r1.70 --- step.py 6 Sep 2005 22:07:07 -0000 1.69 +++ step.py 20 Oct 2005 22:32:48 -0000 1.70 @@ -1323,6 +1323,7 @@ slavever = self.slaveVersion("darcs") assert slavever, "slave is too old, does not know about darcs" + # TODO: 0.6.6 slaves can't handle args['revision'] if self.repourl: self.args['repourl'] = self.repourl else: @@ -1366,6 +1367,7 @@ """ name = "arch" + # TODO: slaves >0.6.6 will accept args['build-config'], so use it def __init__(self, url, version, archive=None, **kwargs): """ @@ -1441,6 +1443,7 @@ """Bazaar is an alternative client for Arch repositories. baz is mostly compatible with tla, but archive registration is slightly different.""" + # TODO: slaves >0.6.6 will accept args['build-config'], so use it def __init__(self, url, version, archive, **kwargs): """ From warner at users.sourceforge.net Thu Oct 20 22:32:50 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:32:50 +0000 Subject: [Buildbot-commits] buildbot/docs/examples twisted_master.cfg,1.30,1.31 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/examples In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11143/docs/examples Modified Files: twisted_master.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-346 Creator: Brian Warner update sample master.cfg files to use Schedulers Index: twisted_master.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/examples/twisted_master.cfg,v retrieving revision 1.30 retrieving revision 1.31 diff -u -d -r1.30 -r1.31 --- twisted_master.cfg 19 Oct 2005 07:06:01 -0000 1.30 +++ twisted_master.cfg 20 Oct 2005 22:32:48 -0000 1.31 @@ -15,6 +15,7 @@ from buildbot.changes.freshcvs import FreshCVSSource from buildbot.changes.freshcvsmail import FCMaildirSource from buildbot.changes.pb import PBChangeSource +from buildbot.scheduler import Scheduler from buildbot.process.process_twisted import \ QuickTwistedBuildFactory, \ FullTwistedBuildFactory, \ @@ -69,6 +70,17 @@ source = PBChangeSource(prefix="trunk") c['sources'].append(source) +## configure the schedulers +s_quick = Scheduler(name="quick", branch=None, treeStableTimer=30, + builderNames=["quick"]) +s_all = Scheduler(name="all", branch=None, treeStableTimer=5*60, + builderNames=["full-2.2", "full-2.3", "full-2.4", + "reactors", "OS-X", "win32", + #"freebsd", + #"threadless", + ]) +c['schedulers'] = [s_quick, s_all] + ## configure the builders From warner at users.sourceforge.net Thu Oct 20 22:32:50 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:32:50 +0000 Subject: [Buildbot-commits] buildbot/buildbot/scripts sample.cfg,1.5,1.6 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/scripts In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11143/buildbot/scripts Modified Files: sample.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-346 Creator: Brian Warner update sample master.cfg files to use Schedulers Index: sample.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/sample.cfg,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- sample.cfg 11 May 2005 23:25:25 -0000 1.5 +++ sample.cfg 20 Oct 2005 22:32:48 -0000 1.6 @@ -1,3 +1,4 @@ +# -*- python -*- # This is a sample buildmaster config file. It must be installed as # 'master.cfg' in your buildmaster's base directory (although the filename @@ -9,6 +10,7 @@ import os.path from buildbot.changes.freshcvs import FreshCVSSource +from buildbot.scheduler import Scheduler from buildbot.process import step, factory from buildbot.status import html s = factory.s @@ -41,6 +43,14 @@ #fc_source = FreshCVSSource("cvs.example.com", 4519, "foo", "bar") #c['sources'].append(fc_source) +## configure the Schedulers + +c['schedulers'] = [] +c['schedulers'].append(Scheduler(name="all", branch=None, + treeStableTimer=2*60, + builderNames=["buildbot-full"])) + + # the 'builders' list defines the Builders. Each one is configured with a # dictionary, using the following keys: From warner at users.sourceforge.net Thu Oct 20 22:32:50 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Thu, 20 Oct 2005 22:32:50 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status mail.py,1.20,1.21 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11143/buildbot/status Modified Files: mail.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-346 Creator: Brian Warner update sample master.cfg files to use Schedulers Index: mail.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/mail.py,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- mail.py 19 Oct 2005 05:59:34 -0000 1.20 +++ mail.py 20 Oct 2005 22:32:48 -0000 1.21 @@ -232,7 +232,7 @@ text += "The Buildbot has detected a new failure of %s.\n" % name buildurl = self.status.getURLForThing(build) if buildurl: - text += "Full details are available at: %s\n" % buildurl + text += "Full details are available at:\n %s\n" % buildurl text += "\n" url = self.status.getBuildbotURL() From warner at users.sourceforge.net Fri Oct 21 08:03:41 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:03:41 +0000 Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.35,1.36 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28628/buildbot Modified Files: interfaces.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-348 Creator: Brian Warner add HTML/IRC control over build-on-branch and build-revision * buildbot/status/words.py (IrcStatusBot.command_FORCE): add control over --branch and --revision, not that they are always legal to provide * buildbot/status/html.py (StatusResourceBuilder.force): same (StatusResourceBuild.body): display SourceStamp components * buildbot/scripts/runner.py (ForceOptions): option parser for the IRC 'force' command, so it can be shared with an eventual command-line-tool 'buildbot force' mode. * buildbot/test/test_runner.py (Options.testForceOptions): test it Index: interfaces.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v retrieving revision 1.35 retrieving revision 1.36 diff -u -d -r1.35 -r1.36 --- interfaces.py 20 Oct 2005 22:25:29 -0000 1.35 +++ interfaces.py 21 Oct 2005 08:03:39 -0000 1.36 @@ -315,6 +315,9 @@ # TODO: it should be possible to expire the patch but still remember # that the build was r123+something. + # TODO: change this to return the actual SourceStamp instance, and + # remove getChanges() + def getChanges(): """Return a list of Change objects which represent which source changes went into the build.""" From warner at users.sourceforge.net Fri Oct 21 08:03:41 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:03:41 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.517,1.518 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28628 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-348 Creator: Brian Warner add HTML/IRC control over build-on-branch and build-revision * buildbot/status/words.py (IrcStatusBot.command_FORCE): add control over --branch and --revision, not that they are always legal to provide * buildbot/status/html.py (StatusResourceBuilder.force): same (StatusResourceBuild.body): display SourceStamp components * buildbot/scripts/runner.py (ForceOptions): option parser for the IRC 'force' command, so it can be shared with an eventual command-line-tool 'buildbot force' mode. * buildbot/test/test_runner.py (Options.testForceOptions): test it Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.517 retrieving revision 1.518 diff -u -d -r1.517 -r1.518 --- ChangeLog 20 Oct 2005 22:32:48 -0000 1.517 +++ ChangeLog 21 Oct 2005 08:03:39 -0000 1.518 @@ -1,3 +1,16 @@ +2005-10-21 Brian Warner + + * buildbot/status/words.py (IrcStatusBot.command_FORCE): add + control over --branch and --revision, not that they are always + legal to provide + * buildbot/status/html.py (StatusResourceBuilder.force): same + (StatusResourceBuild.body): display SourceStamp components + + * buildbot/scripts/runner.py (ForceOptions): option parser for the + IRC 'force' command, so it can be shared with an eventual + command-line-tool 'buildbot force' mode. + * buildbot/test/test_runner.py (Options.testForceOptions): test it + 2005-10-20 Brian Warner * buildbot/status/mail.py (MailNotifier.buildMessage): reformat From warner at users.sourceforge.net Fri Oct 21 08:03:41 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:03:41 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_runner.py,1.10,1.11 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28628/buildbot/test Modified Files: test_runner.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-348 Creator: Brian Warner add HTML/IRC control over build-on-branch and build-revision * buildbot/status/words.py (IrcStatusBot.command_FORCE): add control over --branch and --revision, not that they are always legal to provide * buildbot/status/html.py (StatusResourceBuilder.force): same (StatusResourceBuild.body): display SourceStamp components * buildbot/scripts/runner.py (ForceOptions): option parser for the IRC 'force' command, so it can be shared with an eventual command-line-tool 'buildbot force' mode. * buildbot/test/test_runner.py (Options.testForceOptions): test it Index: test_runner.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_runner.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- test_runner.py 31 Aug 2005 01:51:25 -0000 1.10 +++ test_runner.py 21 Oct 2005 08:03:38 -0000 1.11 @@ -2,8 +2,8 @@ # this file tests the 'buildbot' command, with its various sub-commands from twisted.trial import unittest -from twisted.python import runtime -import os, os.path, shutil +from twisted.python import runtime, usage +import os, os.path, shutil, shlex from buildbot.scripts import runner, tryclient @@ -41,6 +41,32 @@ os.makedirs(os.sep.join(["nothome", "dir1"])) self.check(["nothome", "dir1"], None) + def doForce(self, args, expected): + o = runner.ForceOptions() + o.parseOptions(args) + self.failUnlessEqual(o.keys(), expected.keys()) + for k in o.keys(): + self.failUnlessEqual(o[k], expected[k], + "[%s] got %s instead of %s" % (k, o[k], + expected[k])) + + def testForceOptions(self): + exp = {"builder": "b1", "reason": "reason", + "branch": None, "revision": None} + self.doForce(shlex.split("b1 reason"), exp) + self.doForce(shlex.split("b1 'reason'"), exp) + self.failUnlessRaises(usage.UsageError, self.doForce, + shlex.split("--builder b1 'reason'"), exp) + self.doForce(shlex.split("--builder b1 --reason reason"), exp) + self.doForce(shlex.split("--builder b1 --reason 'reason'"), exp) + self.doForce(shlex.split("--builder b1 --reason \"reason\""), exp) + + exp['reason'] = "longer reason" + self.doForce(shlex.split("b1 'longer reason'"), exp) + self.doForce(shlex.split("b1 longer reason"), exp) + self.doForce(shlex.split("--reason 'longer reason' b1"), exp) + + class Create(unittest.TestCase): def failUnlessIn(self, substring, string, msg=None): # trial provides a version of this that requires python-2.3 to test From warner at users.sourceforge.net Fri Oct 21 08:03:41 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:03:41 +0000 Subject: [Buildbot-commits] buildbot/buildbot/scripts runner.py,1.36,1.37 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/scripts In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28628/buildbot/scripts Modified Files: runner.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-348 Creator: Brian Warner add HTML/IRC control over build-on-branch and build-revision * buildbot/status/words.py (IrcStatusBot.command_FORCE): add control over --branch and --revision, not that they are always legal to provide * buildbot/status/html.py (StatusResourceBuilder.force): same (StatusResourceBuild.body): display SourceStamp components * buildbot/scripts/runner.py (ForceOptions): option parser for the IRC 'force' command, so it can be shared with an eventual command-line-tool 'buildbot force' mode. * buildbot/test/test_runner.py (Options.testForceOptions): test it Index: runner.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/runner.py,v retrieving revision 1.36 retrieving revision 1.37 diff -u -d -r1.36 -r1.37 --- runner.py 31 Aug 2005 02:26:21 -0000 1.36 +++ runner.py 21 Oct 2005 08:03:39 -0000 1.37 @@ -563,6 +563,26 @@ return d +class ForceOptions(usage.Options): + optParameters = [ + ["builder", None, None, "which Builder to start"], + ["branch", None, None, "which branch to build"], + ["revision", None, None, "which revision to build"], + ["reason", None, None, "the reason for starting the build"], + ] + + def parseArgs(self, *args): + args = list(args) + if len(args) > 0: + if self['builder'] is not None: + raise usage.UsageError("--builder provided in two ways") + self['builder'] = args.pop(0) + if len(args) > 0: + if self['reason'] is not None: + raise usage.UsageError("--reason provided in two ways") + self['reason'] = " ".join(args) + + class TryOptions(usage.Options): optParameters = [ ["connect", "c", None, @@ -655,8 +675,9 @@ ['statusgui', None, StatusClientOptions, "Display a small window showing current builder status"], - ['try', None, TryOptions, - "Run a build with your local changes"], + #['force', None, ForceOptions, "Run a build"], + ['try', None, TryOptions, "Run a build with your local changes"], + ['tryserver', None, TryServerOptions, "buildmaster-side 'try' support function, not for users"], From warner at users.sourceforge.net Fri Oct 21 08:03:41 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:03:41 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.71,1.72 words.py,1.43,1.44 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28628/buildbot/status Modified Files: html.py words.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-348 Creator: Brian Warner add HTML/IRC control over build-on-branch and build-revision * buildbot/status/words.py (IrcStatusBot.command_FORCE): add control over --branch and --revision, not that they are always legal to provide * buildbot/status/html.py (StatusResourceBuilder.force): same (StatusResourceBuild.body): display SourceStamp components * buildbot/scripts/runner.py (ForceOptions): option parser for the IRC 'force' command, so it can be shared with an eventual command-line-tool 'buildbot force' mode. * buildbot/test/test_runner.py (Options.testForceOptions): test it Index: html.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v retrieving revision 1.71 retrieving revision 1.72 diff -u -d -r1.71 -r1.72 --- html.py 20 Oct 2005 22:25:29 -0000 1.71 +++ html.py 21 Oct 2005 08:03:39 -0000 1.72 @@ -4,7 +4,7 @@ from twisted.python import log, components from twisted.python.util import sibpath -import urllib +import urllib, re from twisted.internet import defer, reactor from twisted.web.resource import Resource @@ -309,6 +309,23 @@ % (self.status.getURLForThing(b.getBuilder()), urllib.quote(b.getBuilder().getName()), b.getNumber(), html.escape(b.getReason()))) + + branch, revision, patch = b.getSourceStamp() + data += "

      SourceStamp:

      \n" + data += "
        \n" + if branch: + data += "
      • Branch: %s
      • \n" % html.escape(branch) + if revision: + data += "
      • Revision: %s
      • \n" % html.escape(revision) + if patch: + data += "
      • Patch: YES
      • \n" # TODO: provide link to .diff + if b.getChanges(): + data += "
      • Changes: see below
      • \n" + if (branch is None and revision is None and patch is None + and not b.getChanges()): + data += "
      • build of most recent revision
      • \n" + data += "
      \n" + if b.isFinished(): data += "

      Buildslave: %s

      \n" % html.escape(b.getSlavename()) data += "

      Results:

      \n" @@ -442,6 +459,10 @@ "") + make_row("Reason for build:", "") + + make_row("Branch to build:", + "") + + make_row("Revision to build:", + "") + """ @@ -467,25 +488,43 @@ def force(self, request): name = request.args.get("username", [""])[0] reason = request.args.get("comments", [""])[0] + branch = request.args.get("branch", [""])[0] + revision = request.args.get("revision", [""])[0] + r = "The web-page 'force build' button was pressed by '%s': %s\n" \ % (name, reason) - log.msg("web forcebuild of builder '%s'" % self.builder.name) - if self.control: - # TODO: if we can authenticate that a particular User pushed the - # button, use their name instead of None, so they'll be informed of - # the results. - branch = None - s = SourceStamp(branch=branch) - req = BuildRequest(r, s, self.builder.getName()) - try: - self.control.requestBuildSoon(req) - except interfaces.NoSlaveError: - # TODO: tell the web user that their request could not be - # honored - pass - else: + log.msg("web forcebuild of builder '%s', branch='%s', revision='%s'" + % (self.builder.name, branch, revision)) + + if not self.control: # TODO: tell the web user that their request was denied log.msg("but builder control is disabled") + return Redirect("..") + + # keep weird stuff out of the branch and revision strings. TODO: + # centralize this somewhere. + if not re.match(r'^[\w\.\-\/]*$', branch): + log.msg("bad branch '%s'" % branch) + return Redirect("..") + if not re.match(r'^[\w\.\-\/]*$', revision): + log.msg("bad revision '%s'" % revision) + return Redirect("..") + if branch == "": + branch = None + if revision == "": + revision = None + + # TODO: if we can authenticate that a particular User pushed the + # button, use their name instead of None, so they'll be informed of + # the results. + s = SourceStamp(branch=branch, revision=revision) + req = BuildRequest(r, s, self.builder.getName()) + try: + self.control.requestBuildSoon(req) + except interfaces.NoSlaveError: + # TODO: tell the web user that their request could not be + # honored + pass return Redirect("..") def ping(self, request): Index: words.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/words.py,v retrieving revision 1.43 retrieving revision 1.44 diff -u -d -r1.43 -r1.44 --- words.py 20 Oct 2005 22:25:29 -0000 1.43 +++ words.py 21 Oct 2005 08:03:39 -0000 1.44 @@ -3,7 +3,7 @@ # code to deliver build status through twisted.words (instant messaging # protocols: irc, etc) -import traceback, StringIO, re +import traceback, StringIO, re, shlex from twisted.internet import protocol, reactor try: @@ -21,6 +21,7 @@ from buildbot.process.base import BuildRequest from buildbot.status import base from buildbot.status.builder import SUCCESS, WARNINGS, FAILURE, EXCEPTION +from buildbot.scripts.runner import ForceOptions class UsageError(ValueError): def __init__(self, string = "Invalid usage", *more): @@ -287,11 +288,27 @@ self.reply(reply, "Build details are at %s" % buildurl) def command_FORCE(self, user, reply, args): - args = args.split(None, 2) - if len(args) < 3 or args[0] != 'build': - raise UsageError, "try 'force build WHICH '" - which = args[1] - reason = args[2] + args = shlex.split(args) + if args.pop(0) != "build": + raise UsageError("try 'force build WHICH '") + opts = ForceOptions() + opts.parseOptions(args) + + which = opts['builder'] + branch = opts['branch'] + revision = opts['revision'] + reason = opts['reason'] + + # keep weird stuff out of the branch and revision strings. TODO: + # centralize this somewhere. + if branch and not re.match(r'^[\w\.\-\/]*$', branch): + log.msg("bad branch '%s'" % branch) + self.reply(reply, "sorry, bad branch '%s'" % branch) + return + if revision and not re.match(r'^[\w\.\-\/]*$', revision): + log.msg("bad revision '%s'" % revision) + self.reply(reply, "sorry, bad revision '%s'" % revision) + return bc = self.getControl(which) @@ -303,8 +320,7 @@ r = "forced: by IRC user <%s>: %s" % (user, reason) # TODO: maybe give certain users the ability to request builds of # certain branches - branch = None - s = SourceStamp(branch=branch) + s = SourceStamp(branch=branch, revision=revision) req = BuildRequest(r, s, which) try: bc.requestBuildSoon(req) From warner at users.sourceforge.net Fri Oct 21 08:14:54 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:14:54 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.518,1.519 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31540 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-350 Creator: Brian Warner enable build-on-branch for the Twisted buildbot * buildbot/process/process_twisted.py: rework all BuildFactory classes to take a 'source' step as an argument, instead of building up the SVN instance in the factory. * docs/examples/twisted_master.cfg: enable build-on-branch by providing a base_url and default_branch Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.518 retrieving revision 1.519 diff -u -d -r1.518 -r1.519 --- ChangeLog 21 Oct 2005 08:03:39 -0000 1.518 +++ ChangeLog 21 Oct 2005 08:14:52 -0000 1.519 @@ -1,5 +1,11 @@ 2005-10-21 Brian Warner + * buildbot/process/process_twisted.py: rework all BuildFactory + classes to take a 'source' step as an argument, instead of + building up the SVN instance in the factory. + * docs/examples/twisted_master.cfg: enable build-on-branch by + providing a base_url and default_branch + * buildbot/status/words.py (IrcStatusBot.command_FORCE): add control over --branch and --revision, not that they are always legal to provide From warner at users.sourceforge.net Fri Oct 21 08:14:54 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:14:54 +0000 Subject: [Buildbot-commits] buildbot/docs/examples twisted_master.cfg,1.31,1.32 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/examples In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31540/docs/examples Modified Files: twisted_master.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-350 Creator: Brian Warner enable build-on-branch for the Twisted buildbot * buildbot/process/process_twisted.py: rework all BuildFactory classes to take a 'source' step as an argument, instead of building up the SVN instance in the factory. * docs/examples/twisted_master.cfg: enable build-on-branch by providing a base_url and default_branch Index: twisted_master.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/examples/twisted_master.cfg,v retrieving revision 1.31 retrieving revision 1.32 diff -u -d -r1.31 -r1.32 --- twisted_master.cfg 20 Oct 2005 22:32:48 -0000 1.31 +++ twisted_master.cfg 21 Oct 2005 08:14:52 -0000 1.32 @@ -85,13 +85,26 @@ ## configure the builders svnurl = "svn://svn.twistedmatrix.com/svn/Twisted/trunk" +# for build-on-branch, we use these instead +base_url = "svn://svn.twistedmatrix.com/svn/Twisted/" +default_branch = "trunk" +#source_update = s(step.SVN, svnurl=svnurl, mode="update") +#source_copy = s(step.SVN, svnurl=svnurl, mode="copy") +#source_export = s(step.SVN, svnurl=svnurl, mode="export") +source_update = s(step.SVN, base_url=base_url, default_branch=default_branch, + mode="update") +source_copy = s(step.SVN, base_url=base_url, default_branch=default_branch, + mode="copy") +source_export = s(step.SVN, base_url=base_url, default_branch=default_branch, + mode="export") + builders = [] b1 = {'name': "quick", 'slavename': "bot1", 'builddir': "quick", - 'factory': QuickTwistedBuildFactory(svnurl, + 'factory': QuickTwistedBuildFactory(source_update, python=["python2.2", "python2.3"]), } builders.append(b1) @@ -99,7 +112,7 @@ b22 = {'name': "full-2.2", 'slavename': "bot-exarkun", 'builddir': "full2.2", - 'factory': FullTwistedBuildFactory(svnurl, + 'factory': FullTwistedBuildFactory(source_copy, python="python2.2", processDocs=0), } @@ -112,7 +125,7 @@ b23 = {'name': "full-2.3", 'slavename': "bot2", 'builddir': "full2.3", - 'factory': FullTwistedBuildFactory(svnurl, + 'factory': FullTwistedBuildFactory(source_copy, python=["python2.3", "-Wall"], # use -Werror soon compileOpts=b23compile_opts, @@ -128,7 +141,7 @@ b24 = {'name': "full-2.4", 'slavename': "bot-exarkun", 'builddir': "full2.4", - 'factory': FullTwistedBuildFactory(svnurl, + 'factory': FullTwistedBuildFactory(source_copy, python=["python2.4", "-Wall"], # use -Werror soon compileOpts=b24compile_opts, @@ -139,7 +152,7 @@ b3 = {'name': "debuild", 'slavename': "bot2", 'builddir': "debuild", - 'factory': TwistedDebsBuildFactory(svnurl, + 'factory': TwistedDebsBuildFactory(source_export, python="python2.2"), } # debuild is offline while we figure out how to build 2.0 .debs from SVN @@ -149,7 +162,7 @@ b4 = {'name': "reactors", 'slavename': "bot2", 'builddir': "reactors", - 'factory': TwistedReactorsBuildFactory(svnurl, + 'factory': TwistedReactorsBuildFactory(source_copy, python="python2.3", reactors=reactors), } @@ -159,7 +172,7 @@ b23osx = {'name': "OS-X", 'slavename': "bot-jerub", 'builddir': "OSX-full2.4", - 'factory': TwistedReactorsBuildFactory(svnurl, + 'factory': TwistedReactorsBuildFactory(source_copy, python="python2.4", reactors=["default", # "cf", @@ -172,7 +185,7 @@ b22w32 = {'name': "win32", 'slavename': "bot-w32", 'builddir': "W32-full2.2", - 'factory': TwistedReactorsBuildFactory(svnurl, + 'factory': TwistedReactorsBuildFactory(source_copy, python="python", compileOpts2=["-c","mingw32"], reactors=["default", @@ -185,7 +198,7 @@ b23bsd = {'name': "freebsd", 'slavename': "bot-suszko", 'builddir': "bsd-full2.2", - 'factory': TwistedReactorsBuildFactory(svnurl, + 'factory': TwistedReactorsBuildFactory(source_copy, python="python2.3", reactors=["default", "kqueue", @@ -196,7 +209,7 @@ b24threadless = {'name': 'threadless', 'slavename': 'bot-threadless', 'builddir': 'debian-threadless-2.4', - 'factory': TwistedReactorsBuildFactory(svnurl, + 'factory': TwistedReactorsBuildFactory(source_copy, python='python', reactors=['default'])} #builders.append(b24threadless) From warner at users.sourceforge.net Fri Oct 21 08:14:54 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Fri, 21 Oct 2005 08:14:54 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process process_twisted.py,1.39,1.40 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31540/buildbot/process Modified Files: process_twisted.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-350 Creator: Brian Warner enable build-on-branch for the Twisted buildbot * buildbot/process/process_twisted.py: rework all BuildFactory classes to take a 'source' step as an argument, instead of building up the SVN instance in the factory. * docs/examples/twisted_master.cfg: enable build-on-branch by providing a base_url and default_branch Index: process_twisted.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/process_twisted.py,v retrieving revision 1.39 retrieving revision 1.40 diff -u -d -r1.39 -r1.40 --- process_twisted.py 18 Oct 2005 06:29:56 -0000 1.39 +++ process_twisted.py 21 Oct 2005 08:14:52 -0000 1.40 @@ -28,7 +28,6 @@ class TwistedBaseFactory(BuildFactory): buildClass = TwistedBuild - mode = "update" # bin/trial expects its parent directory to be named "Twisted": it uses # this to add the local tree to PYTHONPATH during tests workdir = "Twisted" @@ -42,11 +41,11 @@ treeStableTimer = 30 useProgress = 0 - def __init__(self, svnurl, python="python"): + def __init__(self, source, python="python"): if type(python) is str: python = [python] self.steps = [] - self.steps.append(s(step.SVN, svnurl=svnurl, mode=self.mode)) + self.steps.append(source) self.steps.append(s(HLint, python=python[0])) self.steps.append(s(RemovePYCs)) for p in python: @@ -59,13 +58,12 @@ class FullTwistedBuildFactory(TwistedBaseFactory): treeStableTimer = 5*60 - mode = "copy" - def __init__(self, svnurl, python="python", + def __init__(self, source, python="python", processDocs=False, runTestsRandomly=False, compileOpts=[], compileOpts2=[]): self.steps = [] - self.steps.append(s(step.SVN, svnurl=svnurl, mode=self.mode)) + self.steps.append(source) if processDocs: self.steps.append(s(ProcessDocs)) @@ -84,20 +82,20 @@ class TwistedDebsBuildFactory(TwistedBaseFactory): treeStableTimer = 10*60 - def __init__(self, svnurl, python="python"): + def __init__(self, source, python="python"): self.steps = [] - self.steps.append(s(step.SVN, svnurl=svnurl, mode="export")) + self.steps.append(source) self.steps.append(s(ProcessDocs, haltOnFailure=True)) self.steps.append(s(BuildDebs, warnOnWarnings=True)) class TwistedReactorsBuildFactory(TwistedBaseFactory): treeStableTimer = 5*60 - def __init__(self, svnurl, + def __init__(self, source, python="python", compileOpts=[], compileOpts2=[], reactors=None): self.steps = [] - self.steps.append(s(step.SVN, svnurl=svnurl, mode="copy")) + self.steps.append(source) if type(python) == str: python = [python] From warner at users.sourceforge.net Sat Oct 22 21:10:07 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 21:10:07 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.519,1.520 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14190 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-352 Creator: Brian Warner remove old freshcvs stuff from twisted_master.cfg * buildbot/status/words.py (IrcStatusBot.command_FORCE): note that the 'force' command requires python2.3, for the shlex.split method * docs/examples/twisted_master.cfg: remove old freshcvs stuff, since we don't use it anymore. The Twisted buildbot uses a PBChangeSource now. Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.519 retrieving revision 1.520 diff -u -d -r1.519 -r1.520 --- ChangeLog 21 Oct 2005 08:14:52 -0000 1.519 +++ ChangeLog 22 Oct 2005 21:10:04 -0000 1.520 @@ -1,3 +1,12 @@ +2005-10-22 Brian Warner + + * buildbot/status/words.py (IrcStatusBot.command_FORCE): note that + the 'force' command requires python2.3, for the shlex.split method + + * docs/examples/twisted_master.cfg: remove old freshcvs stuff, + since we don't use it anymore. The Twisted buildbot uses a + PBChangeSource now. + 2005-10-21 Brian Warner * buildbot/process/process_twisted.py: rework all BuildFactory From warner at users.sourceforge.net Sat Oct 22 21:10:07 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 21:10:07 +0000 Subject: [Buildbot-commits] buildbot/docs/examples twisted_master.cfg,1.32,1.33 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/examples In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14190/docs/examples Modified Files: twisted_master.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-352 Creator: Brian Warner remove old freshcvs stuff from twisted_master.cfg * buildbot/status/words.py (IrcStatusBot.command_FORCE): note that the 'force' command requires python2.3, for the shlex.split method * docs/examples/twisted_master.cfg: remove old freshcvs stuff, since we don't use it anymore. The Twisted buildbot uses a PBChangeSource now. Index: twisted_master.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/examples/twisted_master.cfg,v retrieving revision 1.32 retrieving revision 1.33 diff -u -d -r1.32 -r1.33 --- twisted_master.cfg 21 Oct 2005 08:14:52 -0000 1.32 +++ twisted_master.cfg 22 Oct 2005 21:10:05 -0000 1.33 @@ -12,10 +12,10 @@ import os.path from buildbot import master -from buildbot.changes.freshcvs import FreshCVSSource -from buildbot.changes.freshcvsmail import FCMaildirSource from buildbot.changes.pb import PBChangeSource from buildbot.scheduler import Scheduler +from buildbot.process import step +from buildbot.process.factory import s from buildbot.process.process_twisted import \ QuickTwistedBuildFactory, \ FullTwistedBuildFactory, \ @@ -30,8 +30,6 @@ # I set really=False when testing this configuration at home really = True -useFreshCVS = False -useMaildir = False usePBChangeSource = True @@ -40,21 +38,6 @@ c['bots'].append((bot, private.bot_passwords[bot])) c['sources'] = [] -if useFreshCVS: - if really: - fc_source = FreshCVSSource("cvs.twistedmatrix.com", - private.freshcvs_port, - private.freshcvs_perspective, - private.freshcvs_password, - "cvstoys.notify", - prefix="Twisted/") - c['sources'].append(fc_source) - -if useMaildir: - # use with: echo "$BASEDIR/cvsmail/" >~/.qmail - source = FCMaildirSource(os.path.join(basedir, "cvsmail"), - prefix="Twisted/") - c['sources'].append(source) # the Twisted buildbot currently uses the contrib/svn_buildbot.py script. # This makes a TCP connection to the ChangeMaster service to push Changes From warner at users.sourceforge.net Sat Oct 22 21:10:07 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 21:10:07 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status words.py,1.44,1.45 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14190/buildbot/status Modified Files: words.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-352 Creator: Brian Warner remove old freshcvs stuff from twisted_master.cfg * buildbot/status/words.py (IrcStatusBot.command_FORCE): note that the 'force' command requires python2.3, for the shlex.split method * docs/examples/twisted_master.cfg: remove old freshcvs stuff, since we don't use it anymore. The Twisted buildbot uses a PBChangeSource now. Index: words.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/words.py,v retrieving revision 1.44 retrieving revision 1.45 diff -u -d -r1.44 -r1.45 --- words.py 21 Oct 2005 08:03:39 -0000 1.44 +++ words.py 22 Oct 2005 21:10:05 -0000 1.45 @@ -288,7 +288,7 @@ self.reply(reply, "Build details are at %s" % buildurl) def command_FORCE(self, user, reply, args): - args = shlex.split(args) + args = shlex.split(args) # TODO: this requires python2.3 or newer if args.pop(0) != "build": raise UsageError("try 'force build WHICH '") opts = ForceOptions() From warner at users.sourceforge.net Sat Oct 22 22:42:01 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 22:42:01 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.520,1.521 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32691 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-354 Creator: Brian Warner add some slave-versioning code * buildbot/slave/commands.py (cvs_ver): document new features * buildbot/process/step.py (BuildStep.slaveVersion): document it (BuildStep.slaveVersionNewEnough): more useful utility method * buildbot/test/test_steps.py (Version): start testing it Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.520 retrieving revision 1.521 diff -u -d -r1.520 -r1.521 --- ChangeLog 22 Oct 2005 21:10:04 -0000 1.520 +++ ChangeLog 22 Oct 2005 22:41:59 -0000 1.521 @@ -1,5 +1,11 @@ 2005-10-22 Brian Warner + * buildbot/slave/commands.py (cvs_ver): document new features + + * buildbot/process/step.py (BuildStep.slaveVersion): document it + (BuildStep.slaveVersionNewEnough): more useful utility method + * buildbot/test/test_steps.py (Version): start testing it + * buildbot/status/words.py (IrcStatusBot.command_FORCE): note that the 'force' command requires python2.3, for the shlex.split method From warner at users.sourceforge.net Sat Oct 22 22:42:01 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 22:42:01 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_steps.py,1.14,1.15 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32691/buildbot/test Modified Files: test_steps.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-354 Creator: Brian Warner add some slave-versioning code * buildbot/slave/commands.py (cvs_ver): document new features * buildbot/process/step.py (BuildStep.slaveVersion): document it (BuildStep.slaveVersionNewEnough): more useful utility method * buildbot/test/test_steps.py (Version): start testing it Index: test_steps.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- test_steps.py 19 Jul 2005 23:11:58 -0000 1.14 +++ test_steps.py 22 Oct 2005 22:41:58 -0000 1.15 @@ -24,6 +24,9 @@ from buildbot.process import step, base, factory from buildbot.process.step import ShellCommand #, ShellCommands from buildbot.status import builder +from buildbot.test.runutils import RunMixin +from buildbot.twcompat import maybeWait +from buildbot.slave import commands from twisted.python import log #log.startLogging(sys.stdout) @@ -167,3 +170,65 @@ req = base.BuildRequest("reason", SourceStamp()) b = f.newBuild([req]) #for s in b.steps: print s.name + +class VersionCheckingStep(step.BuildStep): + def start(self): + # give our test a chance to run. It is non-trivial for a buildstep to + # claw its way back out to the test case which is currently running. + master = self.build.builder.botmaster.parent + checker = master._checker + checker(self) + # then complete + self.finished(step.SUCCESS) + +version_config = """ +from buildbot.process import factory, step +from buildbot.test.test_steps import VersionCheckingStep +BuildmasterConfig = c = {} +f1 = factory.BuildFactory([ + factory.s(VersionCheckingStep), + ]) +c['bots'] = [['bot1', 'sekrit']] +c['sources'] = [] +c['schedulers'] = [] +c['builders'] = [{'name':'quick', 'slavename':'bot1', + 'builddir': 'quickdir', 'factory': f1}] +c['slavePortnum'] = 0 +""" + +class Version(RunMixin, unittest.TestCase): + def setUp(self): + RunMixin.setUp(self) + self.master.loadConfig(version_config) + self.master.startService() + d = self.connectSlave(["quick"]) + return maybeWait(d) + + def doBuild(self, buildername): + br = base.BuildRequest("forced", SourceStamp()) + d = br.waitUntilFinished() + self.control.getBuilder(buildername).requestBuild(br) + return d + + + def checkCompare(self, s): + v = s.slaveVersion("svn", None) + # this insures that we are getting the version correctly + self.failUnlessEqual(s.slaveVersion("svn", None), commands.cvs_ver) + # and that non-existent commands do not provide a version + self.failUnlessEqual(s.slaveVersion("NOSUCHCOMMAND"), None) + # TODO: verify that a <=0.5.0 buildslave (which does not implement + # remote_getCommands) handles oldversion= properly. This requires a + # mutant slave which does not offer that method. + #self.failUnlessEqual(s.slaveVersion("NOSUCHCOMMAND", "old"), "old") + + # now check the comparison functions + self.failUnless(s.slaveVersionNewEnough("svn", commands.cvs_ver)) + self.failUnless(s.slaveVersionNewEnough("svn", "1.1")) + self.failIf(s.slaveVersionNewEnough("svn", commands.cvs_ver + ".1")) + + def testCompare(self): + self.master._checker = self.checkCompare + d = self.doBuild("quick") + return maybeWait(d) + From warner at users.sourceforge.net Sat Oct 22 22:42:01 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 22:42:01 +0000 Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.39,1.40 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/slave In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32691/buildbot/slave Modified Files: commands.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-354 Creator: Brian Warner add some slave-versioning code * buildbot/slave/commands.py (cvs_ver): document new features * buildbot/process/step.py (BuildStep.slaveVersion): document it (BuildStep.slaveVersionNewEnough): more useful utility method * buildbot/test/test_steps.py (Version): start testing it Index: commands.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v retrieving revision 1.39 retrieving revision 1.40 diff -u -d -r1.39 -r1.40 --- commands.py 16 Aug 2005 20:38:01 -0000 1.39 +++ commands.py 22 Oct 2005 22:41:59 -0000 1.40 @@ -16,6 +16,9 @@ # >=1.17: commands are interruptable # >=1.28: Arch understands 'revision', added Bazaar # >=1.33: Source classes understand 'retry' +# >=1.39: Source classes correctly handle changes in branch (except Git) +# Darcs accepts 'revision' (now all do but Git) (well, and P4Sync) +# Arch/Baz should accept 'build-config' class CommandInterrupted(Exception): pass From warner at users.sourceforge.net Sat Oct 22 22:42:01 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 22:42:01 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.70,1.71 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32691/buildbot/process Modified Files: step.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-354 Creator: Brian Warner add some slave-versioning code * buildbot/slave/commands.py (cvs_ver): document new features * buildbot/process/step.py (BuildStep.slaveVersion): document it (BuildStep.slaveVersionNewEnough): more useful utility method * buildbot/test/test_steps.py (Version): start testing it Index: step.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v retrieving revision 1.70 retrieving revision 1.71 diff -u -d -r1.70 -r1.71 --- step.py 20 Oct 2005 22:32:48 -0000 1.70 +++ step.py 22 Oct 2005 22:41:58 -0000 1.71 @@ -614,8 +614,36 @@ # utility methods that BuildSteps may find useful def slaveVersion(self, command, oldversion=None): + """Return the version number of the given slave command. For the + commands defined in buildbot.slave.commands, this is the value of + 'cvs_ver' at the top of that file. Non-existent commands will return + a value of None. Buildslaves running buildbot-0.5.0 or earlier did + not respond to the version query: commands on those slaves will + return a value of OLDVERSION, so you can distinguish between old + buildslaves and missing commands. + + If you know that <=0.5.0 buildslaves have the command you want (CVS + and SVN existed back then, but none of the other VC systems), then it + makes sense to call this with oldversion='old'. If the command you + want is newer than that, just leave oldversion= unspecified, and the + command will return None for a buildslave that does not implement the + command. + """ return self.build.getSlaveCommandVersion(command, oldversion) + def slaveVersionNewEnough(self, command, minversion): + sv = self.build.getSlaveCommandVersion(command, None) + if sv is None: + return False + # the version we get back is a string form of the CVS version number + # of the slave's buildbot/slave/commands.py, something like 1.39 . + # This might change in the future (I might move away from CVS), but + # if so I'll keep updating that string with suitably-comparable + # values. + if sv.split(".") >= minversion.split("."): + return True + return False + def addLog(self, name): loog = self.step_status.addLog(name) return loog From warner at users.sourceforge.net Sat Oct 22 23:22:12 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 23:22:12 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_runner.py,1.11,1.12 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9958/buildbot/test Modified Files: test_runner.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-356 Creator: Brian Warner handle older slaves better (mostly multiple-branch support) * buildbot/test/test_runner.py (Options.testForceOptions): skip when running under older pythons (<2.3) in which the shlex module doesn't have a 'split' function. * buildbot/process/step.py (ShellCommand.start): make errorMessages= be a list of strings to stuff in the log before the command actually starts. This makes it easier to flag multiple warning messages, e.g. when the Source steps have to deal with an old buildslave. (CVS.startVC): handle slaves that don't handle multiple branches by switching into 'clobber' mode (SVN.startVC): same. Also reject branches without base_url (Darcs.startVC): same. Also reject revision= in older slaves (Arch.checkSlaveVersion): same (just the multiple-branches stuff) (Bazaar.startVC): same, and test for baz separately than for arch Index: test_runner.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_runner.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- test_runner.py 21 Oct 2005 08:03:38 -0000 1.11 +++ test_runner.py 22 Oct 2005 23:22:10 -0000 1.12 @@ -51,6 +51,9 @@ expected[k])) def testForceOptions(self): + if not hasattr(shlex, "split"): + raise unittest.SkipTest("need python>=2.3 for shlex.split") + exp = {"builder": "b1", "reason": "reason", "branch": None, "revision": None} self.doForce(shlex.split("b1 reason"), exp) From warner at users.sourceforge.net Sat Oct 22 23:22:12 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 23:22:12 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.521,1.522 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9958 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-356 Creator: Brian Warner handle older slaves better (mostly multiple-branch support) * buildbot/test/test_runner.py (Options.testForceOptions): skip when running under older pythons (<2.3) in which the shlex module doesn't have a 'split' function. * buildbot/process/step.py (ShellCommand.start): make errorMessages= be a list of strings to stuff in the log before the command actually starts. This makes it easier to flag multiple warning messages, e.g. when the Source steps have to deal with an old buildslave. (CVS.startVC): handle slaves that don't handle multiple branches by switching into 'clobber' mode (SVN.startVC): same. Also reject branches without base_url (Darcs.startVC): same. Also reject revision= in older slaves (Arch.checkSlaveVersion): same (just the multiple-branches stuff) (Bazaar.startVC): same, and test for baz separately than for arch Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.521 retrieving revision 1.522 diff -u -d -r1.521 -r1.522 --- ChangeLog 22 Oct 2005 22:41:59 -0000 1.521 +++ ChangeLog 22 Oct 2005 23:22:10 -0000 1.522 @@ -1,5 +1,21 @@ 2005-10-22 Brian Warner + * buildbot/test/test_runner.py (Options.testForceOptions): skip + when running under older pythons (<2.3) in which the shlex module + doesn't have a 'split' function. + + * buildbot/process/step.py (ShellCommand.start): make + errorMessages= be a list of strings to stuff in the log before the + command actually starts. This makes it easier to flag multiple + warning messages, e.g. when the Source steps have to deal with an + old buildslave. + (CVS.startVC): handle slaves that don't handle multiple branches + by switching into 'clobber' mode + (SVN.startVC): same. Also reject branches without base_url + (Darcs.startVC): same. Also reject revision= in older slaves + (Arch.checkSlaveVersion): same (just the multiple-branches stuff) + (Bazaar.startVC): same, and test for baz separately than for arch + * buildbot/slave/commands.py (cvs_ver): document new features * buildbot/process/step.py (BuildStep.slaveVersion): document it From warner at users.sourceforge.net Sat Oct 22 23:22:12 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sat, 22 Oct 2005 23:22:12 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.71,1.72 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9958/buildbot/process Modified Files: step.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-356 Creator: Brian Warner handle older slaves better (mostly multiple-branch support) * buildbot/test/test_runner.py (Options.testForceOptions): skip when running under older pythons (<2.3) in which the shlex module doesn't have a 'split' function. * buildbot/process/step.py (ShellCommand.start): make errorMessages= be a list of strings to stuff in the log before the command actually starts. This makes it easier to flag multiple warning messages, e.g. when the Source steps have to deal with an old buildslave. (CVS.startVC): handle slaves that don't handle multiple branches by switching into 'clobber' mode (SVN.startVC): same. Also reject branches without base_url (Darcs.startVC): same. Also reject revision= in older slaves (Arch.checkSlaveVersion): same (just the multiple-branches stuff) (Bazaar.startVC): same, and test for baz separately than for arch Index: step.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v retrieving revision 1.71 retrieving revision 1.72 diff -u -d -r1.71 -r1.72 --- step.py 22 Oct 2005 22:41:58 -0000 1.71 +++ step.py 22 Oct 2005 23:22:10 -0000 1.72 @@ -743,7 +743,7 @@ return ["'%s" % words[0], "%s'" % words[1]] return ["'%s" % words[0], "%s" % words[1], "...'"] - def start(self, errorMessage=None): + def start(self, errorMessages=[]): # merge in anything from Build.slaveEnvironment . Earlier steps # (perhaps ones which compile libraries or sub-projects that need to # be referenced by later steps) can add keys to @@ -759,8 +759,8 @@ self.step_status.setColor("yellow") self.step_status.setText(self.describe(False)) loog = self.addLog("log") - if errorMessage: - loog.addHeader(errorMessage) + for em in errorMessages: + loog.addHeader(em) log.msg("ShellCommand.start using log", loog) log.msg(" for cmd", self.cmd) self.cmd.useLog(loog, True) @@ -1200,8 +1200,22 @@ # TODO: figure out why, see if it applies to -r BRANCH self.args['branch'] = None - # accomodate old slaves + # deal with old slaves + warnings = [] slavever = self.slaveVersion("cvs", "old") + if not self.slaveVersionNewEnough("cvs", "1.39"): + # the slave doesn't know to avoid re-using the same sourcedir + # when the branch changes. We have no way of knowing which branch + # the last build used, so if we're using a non-default branch and + # either 'update' or 'copy' modes, it is safer to switch to + # 'clobber' mode. + if branch != None and self.args['mode'] in ("update", "copy"): + m = ("buildslave does not know about multiple branches, " + "switching from mode=%s to mode=clobber to insure " + "we don't build the wrong branch." % self.args['mode']) + warnings.append(m + "\n") + log.msg(m) + self.args['mode'] = "clobber" if slavever == "old": # 0.5.0 if self.args['mode'] == "export": @@ -1214,7 +1228,7 @@ assert not self.args['patch'] # 0.5.0 slave can't do patch self.cmd = LoggedRemoteCommand("cvs", self.args) - ShellCommand.start(self) + ShellCommand.start(self, warnings) class SVN(Source): @@ -1271,29 +1285,45 @@ def startVC(self, branch, revision, patch): # accomodate old slaves - errorMessage = None + warnings = [] slavever = self.slaveVersion("svn", "old") + assert slavever, "slave does not have the 'svn' command" + if not self.slaveVersionNewEnough("svn", "1.39"): + # the slave doesn't know to avoid re-using the same sourcedir + # when the branch changes. We have no way of knowing which branch + # the last build used, so if we're using a non-default branch and + # either 'update' or 'copy' modes, it is safer to switch to + # 'clobber' mode. + if branch != None and self.args['mode'] in ("update", "copy"): + m = ("buildslave does not know about multiple branches, " + "switching from mode=%s to mode=clobber to insure " + "we don't build the wrong branch." % self.args['mode']) + warnings.append(m + "\n") + log.msg(m) + self.args['mode'] = "clobber" + if slavever == "old": # 0.5.0 compatibility if self.args['mode'] in ("clobber", "copy"): # TODO: use some shell commands to make up for the # deficiency, by blowing away the old directory first (thus # forcing a full checkout) - errorMessage = "WARNING: this slave can only do SVN updates" - errorMessage += ", not mode=%s\n" % self.args['mode'] + warnings.append("WARNING: this slave can only do SVN updates" + ", not mode=%s\n" % self.args['mode']) log.msg("WARNING: this slave only does mode=update") assert self.args['mode'] != "export" # more serious self.args['directory'] = self.args['workdir'] if revision is not None: # 0.5.0 can only do HEAD - errorMessage = "WARNING: this slave can only update to HEAD" - errorMessage += ", not revision=%s\n" % revision + warnings.append("WARNING: this slave can only update to " + "HEAD, not revision=%s\n" % revision) log.msg("WARNING: this slave only does -rHEAD") revision = "HEAD" # interprets this key differently assert not patch # 0.5.0 slave can't do patch if self.svnurl: + assert not branch # we need base_url= to use branches self.args['svnurl'] = self.svnurl else: self.args['svnurl'] = self.base_url + branch @@ -1301,7 +1331,7 @@ self.args['patch'] = patch self.cmd = LoggedRemoteCommand("svn", self.args) - ShellCommand.start(self, errorMessage) + ShellCommand.start(self, warnings) class Darcs(Source): @@ -1350,9 +1380,24 @@ def startVC(self, branch, revision, patch): slavever = self.slaveVersion("darcs") assert slavever, "slave is too old, does not know about darcs" + if not self.slaveVersionNewEnough("darcs", "1.39"): + # 0.6.6 slaves can't handle args['revision'] + assert not revision + # the slave doesn't know to avoid re-using the same sourcedir + # when the branch changes. We have no way of knowing which branch + # the last build used, so if we're using a non-default branch and + # either 'update' or 'copy' modes, it is safer to switch to + # 'clobber' mode. + if branch != None and self.args['mode'] in ("update", "copy"): + m = ("buildslave does not know about multiple branches, " + "switching from mode=%s to mode=clobber to insure " + "we don't build the wrong branch." % self.args['mode']) + warnings.append(m + "\m") + log.msg(m) + self.args['mode'] = "clobber" - # TODO: 0.6.6 slaves can't handle args['revision'] if self.repourl: + assert not branch # we need base_url= to use branches self.args['repourl'] = self.repourl else: self.args['repourl'] = self.base_url + branch @@ -1444,27 +1489,38 @@ return "base-0" return "patch-%d" % lastChange - def checkSlaveVersion(self): - slavever = self.slaveVersion("arch") + def checkSlaveVersion(self, cmd): + warnings = [] + slavever = self.slaveVersion(cmd) assert slavever, "slave is too old, does not know about arch" # slave 1.28 and later understand 'revision' - oldslave = False - try: - if slavever.startswith("1.") and int(slavever[2:]) < 28: - oldslave = True - except ValueError: - pass - if oldslave: + if not self.slaveVersionNewEnough(cmd, "1.28"): if not self.alwaysUseLatest: - log.msg("warning, slave is too old to use a revision") + log.msg("warning, buildslave is too old to use a revision") + warnings.append("buildslave is too old to use a revision") + + if not self.slaveVersionNewEnough(cmd, "1.39"): + # the slave doesn't know to avoid re-using the same sourcedir + # when the branch changes. We have no way of knowing which branch + # the last build used, so if we're using a non-default branch and + # either 'update' or 'copy' modes, it is safer to switch to + # 'clobber' mode. + if branch != None and self.args['mode'] in ("update", "copy"): + m = ("buildslave does not know about multiple branches, " + "switching from mode=%s to mode=clobber to insure " + "we don't build the wrong branch." % self.args['mode']) + warnings.append(m + "\n") + log.msg(m) + self.args['mode'] = "clobber" + return warnings def startVC(self, branch, revision, patch): self.args['version'] = branch self.args['revision'] = revision self.args['patch'] = patch - self.checkSlaveVersion() + warnings = self.checkSlaveVersion("arch") self.cmd = LoggedRemoteCommand("arch", self.args) - ShellCommand.start(self) + ShellCommand.start(self, warnings) class Bazaar(Arch): @@ -1495,25 +1551,13 @@ 'archive': archive, }) - def checkSlaveVersion(self): - slavever = self.slaveVersion("arch") - assert slavever, "slave is too old, does not know about arch" - # slave 1.28 and later understand baz - oldslave = False - try: - if slavever.startswith("1.") and int(slavever[2:]) < 28: - oldslave = True - except ValueError: - pass - assert not oldslave, "slave is too old, does not know about baz" - def startVC(self, branch, revision, patch): self.args['version'] = branch self.args['revision'] = revision self.args['patch'] = patch - self.checkSlaveVersion() + warnings = self.checkSlaveVersion("bazaar") self.cmd = LoggedRemoteCommand("bazaar", self.args) - ShellCommand.start(self) + ShellCommand.start(self, warnings) class todo_P4(Source): From warner at users.sourceforge.net Sun Oct 23 05:06:04 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:06:04 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.522,1.523 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4351 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-358 Creator: Brian Warner fix some upgrade problems, fix the forgotten-builds problem * buildbot/status/builder.py (BuildStatus): derive from styles.Versioned, fix upgrade of .sourceStamp attribute. Also set the default (i.e. unknown) .slavename to "???" instead of None, since even unknown slavenames need to be printed eventually. (BuilderStatus): also derive from styles.Versioned . More importantly, determine .nextBuildNumber at creation/unpickling time by scanning the directory of saved BuildStatus instances and choosing one larger than the highest-numbered one found. This should fix the problem where random errors during upgrades cause the buildbot to forget about earlier builds. .nextBuildNumber is no longer stored in the pickle. (Status.builderAdded): if we can't unpickle the BuilderStatus, at least log the error. Also call Builder.determineNextBuildNumber once the basedir is set. * buildbot/master.py (BuildMaster.loadChanges): do styles.doUpgrade afterwards, in case I decide to make Changes derived from styles.Versioned some day and forget to make this change later. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.522 retrieving revision 1.523 diff -u -d -r1.522 -r1.523 --- ChangeLog 22 Oct 2005 23:22:10 -0000 1.522 +++ ChangeLog 23 Oct 2005 05:06:02 -0000 1.523 @@ -1,5 +1,26 @@ 2005-10-22 Brian Warner + * buildbot/status/builder.py (BuildStatus): derive from + styles.Versioned, fix upgrade of .sourceStamp attribute. Also set + the default (i.e. unknown) .slavename to "???" instead of None, + since even unknown slavenames need to be printed eventually. + (BuilderStatus): also derive from styles.Versioned . More + importantly, determine .nextBuildNumber at creation/unpickling + time by scanning the directory of saved BuildStatus instances and + choosing one larger than the highest-numbered one found. This + should fix the problem where random errors during upgrades cause + the buildbot to forget about earlier builds. .nextBuildNumber is + no longer stored in the pickle. + (Status.builderAdded): if we can't unpickle the BuilderStatus, + at least log the error. Also call Builder.determineNextBuildNumber + once the basedir is set. + + * buildbot/master.py (BuildMaster.loadChanges): do + styles.doUpgrade afterwards, in case I decide to make Changes + derived from styles.Versioned some day and forget to make this + change later. + + * buildbot/test/test_runner.py (Options.testForceOptions): skip when running under older pythons (<2.3) in which the shlex module doesn't have a 'split' function. From warner at users.sourceforge.net Sun Oct 23 05:06:04 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:06:04 +0000 Subject: [Buildbot-commits] buildbot/buildbot master.py,1.82,1.83 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4351/buildbot Modified Files: master.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-358 Creator: Brian Warner fix some upgrade problems, fix the forgotten-builds problem * buildbot/status/builder.py (BuildStatus): derive from styles.Versioned, fix upgrade of .sourceStamp attribute. Also set the default (i.e. unknown) .slavename to "???" instead of None, since even unknown slavenames need to be printed eventually. (BuilderStatus): also derive from styles.Versioned . More importantly, determine .nextBuildNumber at creation/unpickling time by scanning the directory of saved BuildStatus instances and choosing one larger than the highest-numbered one found. This should fix the problem where random errors during upgrades cause the buildbot to forget about earlier builds. .nextBuildNumber is no longer stored in the pickle. (Status.builderAdded): if we can't unpickle the BuilderStatus, at least log the error. Also call Builder.determineNextBuildNumber once the basedir is set. * buildbot/master.py (BuildMaster.loadChanges): do styles.doUpgrade afterwards, in case I decide to make Changes derived from styles.Versioned some day and forget to make this change later. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: master.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/master.py,v retrieving revision 1.82 retrieving revision 1.83 diff -u -d -r1.82 -r1.83 --- master.py 14 Oct 2005 19:42:39 -0000 1.82 +++ master.py 23 Oct 2005 05:06:02 -0000 1.83 @@ -635,6 +635,7 @@ filename = os.path.join(self.basedir, "changes.pck") try: changes = pickle.load(open(filename, "r")) + styles.doUpgrade() except IOError: log.msg("changes.pck missing, using new one") changes = ChangeMaster() From warner at users.sourceforge.net Sun Oct 23 05:06:05 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:06:05 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.69,1.70 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4351/buildbot/status Modified Files: builder.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-358 Creator: Brian Warner fix some upgrade problems, fix the forgotten-builds problem * buildbot/status/builder.py (BuildStatus): derive from styles.Versioned, fix upgrade of .sourceStamp attribute. Also set the default (i.e. unknown) .slavename to "???" instead of None, since even unknown slavenames need to be printed eventually. (BuilderStatus): also derive from styles.Versioned . More importantly, determine .nextBuildNumber at creation/unpickling time by scanning the directory of saved BuildStatus instances and choosing one larger than the highest-numbered one found. This should fix the problem where random errors during upgrades cause the buildbot to forget about earlier builds. .nextBuildNumber is no longer stored in the pickle. (Status.builderAdded): if we can't unpickle the BuilderStatus, at least log the error. Also call Builder.determineNextBuildNumber once the basedir is set. * buildbot/master.py (BuildMaster.loadChanges): do styles.doUpgrade afterwards, in case I decide to make Changes derived from styles.Versioned some day and forget to make this change later. --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: builder.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v retrieving revision 1.69 retrieving revision 1.70 diff -u -d -r1.69 -r1.70 --- builder.py 16 Oct 2005 06:25:34 -0000 1.69 +++ builder.py 23 Oct 2005 05:06:02 -0000 1.70 @@ -894,11 +894,12 @@ loog.step = self -class BuildStatus: +class BuildStatus(styles.Versioned): if implements: implements(interfaces.IBuildStatus, interfaces.IStatusEvent) else: __implements__ = interfaces.IBuildStatus, interfaces.IStatusEvent + persistenceVersion = 1 source = None reason = None @@ -911,7 +912,7 @@ text = [] color = None results = None - slavename = None + slavename = "???" # these lists/dicts are defined here so that unserialized instances have # (empty) values. They are set in __init__ to new objects to make sure @@ -1186,7 +1187,7 @@ return filename def __getstate__(self): - d = self.__dict__.copy() + d = styles.Versioned.__getstate__(self) # for now, a serialized Build is always "finished". We will never # save unfinished builds. if not self.finished: @@ -1202,18 +1203,21 @@ return d def __setstate__(self, d): - self.__dict__ = d + styles.Versioned.__setstate__(self, d) # self.builder must be filled in by our parent when loading for step in self.steps: step.build = self self.watchers = [] self.updates = {} self.finishedWatchers = [] - if d.has_key('sourceStamp'): - revision, patch = d['sourceStamp'] - changes = d.get('changes', []) + + def upgradeToVersion1(self): + if hasattr(self, "sourceStamp"): + # the old .sourceStamp attribute wasn't actually very useful + maxChangeNumber, patch = self.sourceStamp + changes = getattr(self, 'changes', []) source = sourcestamp.SourceStamp(branch=None, - revision=revision, + revision=None, patch=patch, changes=changes) self.source = source @@ -1258,7 +1262,7 @@ -class BuilderStatus: +class BuilderStatus(styles.Versioned): """I handle status information for a single process.base.Builder object. That object sends status changes to me (frequently as Events), and I provide them on demand to the various status recipients, like the HTML @@ -1281,6 +1285,7 @@ implements(interfaces.IBuilderStatus) else: __implements__ = interfaces.IBuilderStatus, + persistenceVersion = 1 # these limit the amount of memory we consume, as well as the size of the # main Builder pickle. The Build and LogFile pickles on disk must be @@ -1291,7 +1296,6 @@ category = None currentBigState = "offline" # or idle/waiting/interlocked/building - nextBuildNumber = 0 basedir = None # filled in by our parent def __init__(self, buildername, category=None): @@ -1312,10 +1316,13 @@ self.buildCache = [] # TODO: age builds out of the cache # persistence - # TODO: why am I not using styles.Versioned for this? def __getstate__(self): - d = self.__dict__.copy() + # when saving, don't record transient stuff like what builds are + # currently running, because they won't be there when we start back + # up. Nor do we save self.watchers, nor anything that gets set by our + # parent like .basedir and .status + d = styles.Versioned.__getstate__(self) d['watchers'] = [] del d['buildCache'] for b in self.currentBuilds: @@ -1326,20 +1333,42 @@ del d['currentBigState'] del d['basedir'] del d['status'] + del d['nextBuildNumber'] return d def __setstate__(self, d): - self.__dict__ = d + # when loading, re-initialize the transient stuff. Remember that + # upgradeToVersion1 and such will be called after this finishes. + styles.Versioned.__setstate__(self, d) self.buildCache = [] self.currentBuilds = [] self.pendingBuilds = [] self.watchers = [] - if d.has_key('slavename'): - self.slavenames = [self.slavename] - del self.slavename + self.slavenames = [] # self.basedir must be filled in by our parent # self.status must be filled in by our parent + def upgradeToVersion1(self): + if hasattr(self, 'slavename'): + self.slavenames = [self.slavename] + del self.slavename + if hasattr(self, 'nextBuildNumber'): + del self.nextBuildNumber # determineNextBuildNumber chooses this + + def determineNextBuildNumber(self): + """Scan our directory of saved BuildStatus instances to determine + what our self.nextBuildNumber should be. Set it one larger than the + highest-numbered build we discover. This is called by the top-level + Status object shortly after we are created or loaded from disk. + """ + existing_builds = [int(f) + for f in os.listdir(self.basedir) + if re.match("^\d+$", f)] + if existing_builds: + self.nextBuildNumber = max(existing_builds) + 1 + else: + self.nextBuildNumber = 0 + def saveYourself(self): for b in self.buildCache: if not b.isFinished: @@ -1379,6 +1408,7 @@ filename = os.path.join(self.basedir, "%d" % number) try: build = pickle.load(open(filename, "r")) + styles.doUpgrade() build.builder = self # handle LogFiles from after 0.5.0 and before 0.6.5 build.upgradeLogfiles() @@ -1523,7 +1553,9 @@ number = self.nextBuildNumber self.nextBuildNumber += 1 # TODO: self.saveYourself(), to make sure we don't forget about the - # build number we've just allocated + # build number we've just allocated. This is not quite as important + # as it was before we switch to determineNextBuildNumber, but I think + # it may still be useful to have the new build save itself. s = BuildStatus(self, number) s.waitUntilFinished().addCallback(self._buildFinished) return s @@ -1816,10 +1848,13 @@ builder_status = None try: builder_status = pickle.load(open(filename, "r")) + styles.doUpgrade() except IOError: log.msg("no saved status pickle, creating a new one") except: log.msg("error while loading status pickle, creating a new one") + log.msg("error follows:") + log.err() if not builder_status: builder_status = BuilderStatus(name, category) builder_status.addPointEvent(["builder", "created"]) @@ -1833,6 +1868,8 @@ if not os.path.isdir(builder_status.basedir): os.mkdir(builder_status.basedir) + builder_status.determineNextBuildNumber() + builder_status.setBigState("offline") for t in self.watchers: From warner at users.sourceforge.net Sun Oct 23 05:16:59 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:16:59 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.72,1.73 classic.css,1.1,1.2 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5819/buildbot/status Modified Files: html.py classic.css Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-360 Creator: Brian Warner add countdown timer for upcoming builds, add "waiting" state * buildbot/status/html.py (StatusResourceBuild.body): revision might be numeric, so stringify it before html-escapifying it (CurrentBox.getBox): add a "waiting" state, and show a countdown timer for the upcoming build * buildbot/status/classic.css: add background-color attributes for offline/waiting/building classes Index: classic.css =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/classic.css,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- classic.css 26 Apr 2005 02:43:12 -0000 1.1 +++ classic.css 23 Oct 2005 05:16:57 -0000 1.2 @@ -8,9 +8,18 @@ } /* Activity states */ +.offline { + background-color: red; +} .idle { background-color: white; } +.waiting { + background-color: yellow; +} +.building { + background-color: yellow; +} /* LastBuild, BuildStep states */ .success { Index: html.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v retrieving revision 1.72 retrieving revision 1.73 diff -u -d -r1.72 -r1.73 --- html.py 21 Oct 2005 08:03:39 -0000 1.72 +++ html.py 23 Oct 2005 05:16:57 -0000 1.73 @@ -316,7 +316,7 @@ if branch: data += "
    1. Branch: %s
    2. \n" % html.escape(branch) if revision: - data += "
    3. Revision: %s
    4. \n" % html.escape(revision) + data += "
    5. Revision: %s
    6. \n" % html.escape(str(revision)) if patch: data += "
    7. Patch: YES
    8. \n" # TODO: provide link to .diff if b.getChanges(): @@ -762,7 +762,20 @@ def getBox(self, status): # getState() returns offline, idle, or building state, builds = self.original.getState() - color = "white" + + # look for upcoming builds. We say the state is "waiting" if the + # builder is otherwise idle and there is a scheduler which tells us a + # build will be performed some time in the near future. TODO: this + # functionality used to be in BuilderStatus.. maybe this code should + # be merged back into it. + upcoming = [] + builderName = self.original.getName() + for s in status.getSchedulers(): + if builderName in s.listBuilderNames(): + upcoming.extend(s.getPendingBuildTimes()) + if state == "idle" and upcoming: + state = "waiting" + if state == "building": color = "yellow" text = ["building"] @@ -775,9 +788,14 @@ color = "red" text = ["offline"] elif state == "idle": + color = "white" text = ["idle"] + elif state == "waiting": + color = "yellow" + text = ["waiting"] else: # just in case I add a state and forget to update this + color = "white" text = [state] # TODO: for now, this pending/upcoming stuff is in the "current @@ -789,15 +807,17 @@ pbs = self.original.getPendingBuilds() if pbs: text.append("%d pending" % len(pbs)) - # how about upcoming ones? - upcoming = [] - builderName = self.original.getName() - for s in status.getSchedulers(): - if builderName in s.listBuilderNames(): - upcoming.extend(s.getPendingBuildTimes()) for t in upcoming: text.extend(["next at", - time.strftime("%H:%M:%S", time.localtime(t))]) + time.strftime("%H:%M:%S", time.localtime(t)), + "[%d secs]" % (t - util.now()), + ]) + # TODO: the upcoming-builds box looks like: + # ['waiting', 'next at', '22:14:15', '[86 secs]'] + # while the currently-building box is reversed: + # ['building', 'ETA in', '2 secs', 'at 22:12:50'] + # consider swapping one of these to make them look the same. also + # consider leaving them reversed to make them look different. return Box(text, color=color, class_="Activity " + state) components.registerAdapter(CurrentBox, builder.BuilderStatus, ICurrentBox) From warner at users.sourceforge.net Sun Oct 23 05:16:59 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:16:59 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.523,1.524 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5819 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-360 Creator: Brian Warner add countdown timer for upcoming builds, add "waiting" state * buildbot/status/html.py (StatusResourceBuild.body): revision might be numeric, so stringify it before html-escapifying it (CurrentBox.getBox): add a "waiting" state, and show a countdown timer for the upcoming build * buildbot/status/classic.css: add background-color attributes for offline/waiting/building classes Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.523 retrieving revision 1.524 diff -u -d -r1.523 -r1.524 --- ChangeLog 23 Oct 2005 05:06:02 -0000 1.523 +++ ChangeLog 23 Oct 2005 05:16:57 -0000 1.524 @@ -1,5 +1,12 @@ 2005-10-22 Brian Warner + * buildbot/status/html.py (StatusResourceBuild.body): revision + might be numeric, so stringify it before html-escapifying it + (CurrentBox.getBox): add a "waiting" state, and show a countdown + timer for the upcoming build + * buildbot/status/classic.css: add background-color attributes for + offline/waiting/building classes + * buildbot/status/builder.py (BuildStatus): derive from styles.Versioned, fix upgrade of .sourceStamp attribute. Also set the default (i.e. unknown) .slavename to "???" instead of None, From warner at users.sourceforge.net Sun Oct 23 05:28:49 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:28:49 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_steps.py,1.15,1.16 test_status.py,1.25,1.26 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7576/buildbot/test Modified Files: test_steps.py test_status.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-362 Creator: Brian Warner fix some tests that broke because of the .nextBuildNumber change * buildbot/test/test_steps.py (BuildStep.setUp): set nextBuildNumber so the test passes * buildbot/test/test_status.py (MyBuilder): same Index: test_status.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_status.py,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- test_status.py 19 Oct 2005 06:26:10 -0000 1.25 +++ test_status.py 23 Oct 2005 05:28:46 -0000 1.26 @@ -89,7 +89,7 @@ return self.url class MyBuilder(builder.BuilderStatus): - pass + nextBuildNumber = 0 class MyBuild(builder.BuildStatus): testlogs = [] Index: test_steps.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- test_steps.py 22 Oct 2005 22:41:58 -0000 1.15 +++ test_steps.py 23 Oct 2005 05:28:46 -0000 1.16 @@ -71,6 +71,7 @@ self.builder = FakeBuilder() self.builder_status = builder.BuilderStatus("fakebuilder") self.builder_status.basedir = "test_steps" + self.builder_status.nextBuildNumber = 0 os.mkdir(self.builder_status.basedir) self.build_status = self.builder_status.newBuild() req = base.BuildRequest("reason", SourceStamp()) From warner at users.sourceforge.net Sun Oct 23 05:28:49 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:28:49 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.524,1.525 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7576 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-362 Creator: Brian Warner fix some tests that broke because of the .nextBuildNumber change * buildbot/test/test_steps.py (BuildStep.setUp): set nextBuildNumber so the test passes * buildbot/test/test_status.py (MyBuilder): same Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.524 retrieving revision 1.525 diff -u -d -r1.524 -r1.525 --- ChangeLog 23 Oct 2005 05:16:57 -0000 1.524 +++ ChangeLog 23 Oct 2005 05:28:47 -0000 1.525 @@ -1,5 +1,9 @@ 2005-10-22 Brian Warner + * buildbot/test/test_steps.py (BuildStep.setUp): set + nextBuildNumber so the test passes + * buildbot/test/test_status.py (MyBuilder): same + * buildbot/status/html.py (StatusResourceBuild.body): revision might be numeric, so stringify it before html-escapifying it (CurrentBox.getBox): add a "waiting" state, and show a countdown From warner at users.sourceforge.net Sun Oct 23 05:48:19 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:48:19 +0000 Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.73,1.74 classic.css,1.2,1.3 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/status In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9744/buildbot/status Modified Files: html.py classic.css Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-364 Creator: Brian Warner emit-valid-HTML patch, thanks to Brad Hards * buildbot/status/html.py (HtmlResource): incorporate valid-HTML patch from Brad Hards * buildbot/status/classic.css: same Index: classic.css =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/classic.css,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- classic.css 23 Oct 2005 05:16:57 -0000 1.2 +++ classic.css 23 Oct 2005 05:48:17 -0000 1.3 @@ -29,7 +29,7 @@ background-color: red; } .warnings { - background-color: orange; + background-color: #ff8000; } .exception { background-color: #c000c0; Index: html.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v retrieving revision 1.73 retrieving revision 1.74 diff -u -d -r1.73 -r1.74 --- html.py 23 Oct 2005 05:16:57 -0000 1.73 +++ html.py 23 Oct 2005 05:48:17 -0000 1.74 @@ -142,7 +142,7 @@ class HtmlResource(Resource): css = None - contentType = "text/html" + contentType = "text/html; charset=UTF-8" def render(self, request): data = self.content(request) request.setHeader("content-type", self.contentType) @@ -152,14 +152,22 @@ return data title = "Dummy" def content(self, request): - data = "\n" + self.title + "\n" + data = ('\n' + '\n') + data += "\n" + data += " " + self.title + "\n" if self.css: # TODO: use some sort of relative link up to the root page, so # this css can be used from child pages too - data += ("\n" % "buildbot.css") - data += "\n" + data += (' \n' + % "buildbot.css") + data += "\n" + data += '\n' data += self.body(request) data += "\n" return data From warner at users.sourceforge.net Sun Oct 23 05:48:19 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 05:48:19 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.525,1.526 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9744 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-364 Creator: Brian Warner emit-valid-HTML patch, thanks to Brad Hards * buildbot/status/html.py (HtmlResource): incorporate valid-HTML patch from Brad Hards * buildbot/status/classic.css: same Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.525 retrieving revision 1.526 diff -u -d -r1.525 -r1.526 --- ChangeLog 23 Oct 2005 05:28:47 -0000 1.525 +++ ChangeLog 23 Oct 2005 05:48:17 -0000 1.526 @@ -1,5 +1,9 @@ 2005-10-22 Brian Warner + * buildbot/status/html.py (HtmlResource): incorporate valid-HTML + patch from Brad Hards + * buildbot/status/classic.css: same + * buildbot/test/test_steps.py (BuildStep.setUp): set nextBuildNumber so the test passes * buildbot/test/test_status.py (MyBuilder): same From warner at users.sourceforge.net Sun Oct 23 06:10:09 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Sun, 23 Oct 2005 06:10:09 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_web.py,1.23,1.24 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12411/buildbot/test Modified Files: test_web.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-366 Creator: Brian Warner fix test_web to match valid-HTML changes * buildbot/test/test_web.py (Waterfall): match changes Index: test_web.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_web.py,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- test_web.py 20 Oct 2005 22:25:29 -0000 1.23 +++ test_web.py 23 Oct 2005 06:10:07 -0000 1.24 @@ -209,7 +209,7 @@ def _test_waterfall_1(self, page, port): self.failUnless(page) self.failUnlessIn("current activity", page) - self.failUnlessIn("", page) + self.failUnlessIn("", page) + self.failUnlessIn(" Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12411 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-366 Creator: Brian Warner fix test_web to match valid-HTML changes * buildbot/test/test_web.py (Waterfall): match changes Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.526 retrieving revision 1.527 diff -u -d -r1.526 -r1.527 --- ChangeLog 23 Oct 2005 05:48:17 -0000 1.526 +++ ChangeLog 23 Oct 2005 06:10:07 -0000 1.527 @@ -3,6 +3,7 @@ * buildbot/status/html.py (HtmlResource): incorporate valid-HTML patch from Brad Hards * buildbot/status/classic.css: same + * buildbot/test/test_web.py (Waterfall): match changes * buildbot/test/test_steps.py (BuildStep.setUp): set nextBuildNumber so the test passes From warner at users.sourceforge.net Mon Oct 24 03:48:02 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 03:48:02 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.527,1.528 NEWS,1.45,1.46 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29636 Modified Files: ChangeLog NEWS Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-368 Creator: Brian Warner update NEWS file a little bit Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.527 retrieving revision 1.528 diff -u -d -r1.527 -r1.528 --- ChangeLog 23 Oct 2005 06:10:07 -0000 1.527 +++ ChangeLog 24 Oct 2005 03:47:59 -0000 1.528 @@ -1,3 +1,7 @@ +2005-10-23 Brian Warner + + * NEWS: update + 2005-10-22 Brian Warner * buildbot/status/html.py (HtmlResource): incorporate valid-HTML Index: NEWS =================================================================== RCS file: /cvsroot/buildbot/buildbot/NEWS,v retrieving revision 1.45 retrieving revision 1.46 diff -u -d -r1.45 -r1.46 --- NEWS 3 Sep 2005 19:50:17 -0000 1.45 +++ NEWS 24 Oct 2005 03:48:00 -0000 1.46 @@ -54,6 +54,53 @@ Instructions for developers who want to use 'try' (and the configuration changes necessary to enable its use) are in the user's manual. +** Build-On-Branch + +When suitably configured, the buildbot can be used to build trees from a +variety of related branches. You can set up Schedulers to build a tree using +whichever branch was last changed, or users can request builds of specific +branches through IRC, the web page, or the CLI 'buildbot force' subcommand. + +The IRC 'force' command now takes --branch and --revision arguments (not that +they always make sense). Likewise the HTML 'force build' button now has an +input field for branch and revision. Your build's source-checkout step must +be suitably configured to support this: for SVN it involves giving both a +base URL and a default branch. Other VC systems are configured differently. + +** Multiple slaves per Builder + +You can now attach multiple buildslaves to each Builder. This can provide +redundancy or load-balancing among many machines equally capable of running +the build. To use this, define a key in the Builder specification dictionary +named 'slavenames' with a list of buildslave names (instead of the usual +'slavename' that contains just a single slavename). + +** minor new features + +The IRC and email status-reporting facilities now provide more specific URLs +for particular builds, in addition to the generic buildmaster home page. The +HTML per-build page now has more information. + +The Twisted-specific test classes have been modified to match the argument +syntax preferred by Trial as of Twisted-2.1.0 and newer. The generic trial +steps are still suitable for the Trial that comes with older versions of +Twisted, but may produce deprecation warnings or errors when used with the +latest Trial. + +** bugs fixed + +DNotify, used by the maildir-watching ChangeSources, had problems on some +64-bit systems relating to signed-vs-unsigned constants and the DN_MULTISHOT +flag. A workaround was provided by Brad Hards. + +The web status page should now be valid XHTML, thanks to a patch by Brad +Hards. The charset parameter is specified to be UTF-8, so VC comments, +builder names, etc, should probably all be in UTF-8 to be displayed properly. + +** creeping version dependencies + +The IRC 'force build' command requires python2.3 (for the shlex.split +function). * Release 0.6.6 (23 May 2005) From warner at users.sourceforge.net Mon Oct 24 03:48:19 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 03:48:19 +0000 Subject: [Buildbot-commits] buildbot/docs/examples twisted_master.cfg,1.33,1.34 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/examples In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29694/docs/examples Modified Files: twisted_master.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-369 Creator: Brian Warner update twisted's buildbot config: the OS-X slave now does QT --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: twisted_master.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/examples/twisted_master.cfg,v retrieving revision 1.33 retrieving revision 1.34 diff -u -d -r1.33 -r1.34 --- twisted_master.cfg 22 Oct 2005 21:10:05 -0000 1.33 +++ twisted_master.cfg 24 Oct 2005 03:48:17 -0000 1.34 @@ -158,6 +158,7 @@ 'factory': TwistedReactorsBuildFactory(source_copy, python="python2.4", reactors=["default", + "qt", # "cf", # "threadedselect", ], From warner at users.sourceforge.net Mon Oct 24 03:48:19 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 03:48:19 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.528,1.529 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29694 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-369 Creator: Brian Warner update twisted's buildbot config: the OS-X slave now does QT --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.528 retrieving revision 1.529 diff -u -d -r1.528 -r1.529 --- ChangeLog 24 Oct 2005 03:47:59 -0000 1.528 +++ ChangeLog 24 Oct 2005 03:48:17 -0000 1.529 @@ -1,5 +1,7 @@ 2005-10-23 Brian Warner + * docs/examples/twisted_master.cfg: OS-X slave now does QT + * NEWS: update 2005-10-22 Brian Warner From warner at users.sourceforge.net Mon Oct 24 03:48:33 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 03:48:33 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.529,1.530 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29737 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-370 Creator: Brian Warner fix up the Twisted ProcessDocs buildstep, add TryScheduler * buildbot/process/step_twisted.py (ProcessDocs.createSummary): when creating the list of warning messages, include the line immediately after each WARNING: line, since that's usually where the file and line number wind up. * docs/examples/twisted_master.cfg: OS-X slave now does QT, add a TryScheduler --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.529 retrieving revision 1.530 diff -u -d -r1.529 -r1.530 --- ChangeLog 24 Oct 2005 03:48:17 -0000 1.529 +++ ChangeLog 24 Oct 2005 03:48:30 -0000 1.530 @@ -1,6 +1,12 @@ 2005-10-23 Brian Warner - * docs/examples/twisted_master.cfg: OS-X slave now does QT + * buildbot/process/step_twisted.py (ProcessDocs.createSummary): + when creating the list of warning messages, include the line + immediately after each WARNING: line, since that's usually where + the file and line number wind up. + + * docs/examples/twisted_master.cfg: OS-X slave now does QT, add a + TryScheduler * NEWS: update From warner at users.sourceforge.net Mon Oct 24 03:48:33 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 03:48:33 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step_twisted.py,1.68,1.69 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29737/buildbot/process Modified Files: step_twisted.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-370 Creator: Brian Warner fix up the Twisted ProcessDocs buildstep, add TryScheduler * buildbot/process/step_twisted.py (ProcessDocs.createSummary): when creating the list of warning messages, include the line immediately after each WARNING: line, since that's usually where the file and line number wind up. * docs/examples/twisted_master.cfg: OS-X slave now does QT, add a TryScheduler --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: step_twisted.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step_twisted.py,v retrieving revision 1.68 retrieving revision 1.69 diff -u -d -r1.68 -r1.69 --- step_twisted.py 24 Apr 2005 21:30:25 -0000 1.68 +++ step_twisted.py 24 Oct 2005 03:48:31 -0000 1.69 @@ -616,7 +616,20 @@ def createSummary(self, log): output = log.getText() # hlint warnings are of the format: 'WARNING: file:line:col: stuff + # latex warnings start with "WARNING: LaTeX Warning: stuff", but + # sometimes wrap around to a second line. lines = output.split("\n") + warningLines = [] + wantNext = False + for line in lines: + wantThis = wantNext + wantNext = False + if line.startswith("WARNING: "): + wantThis = True + wantNext = True + if wantThis: + warningLines.append(line) + warningLines = filter(lambda line: line.find("WARNING: ") == 0, lines) if warningLines: self.addCompleteLog("warnings", "".join(warningLines)) From warner at users.sourceforge.net Mon Oct 24 03:48:33 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 03:48:33 +0000 Subject: [Buildbot-commits] buildbot/docs/examples twisted_master.cfg,1.34,1.35 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/examples In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29737/docs/examples Modified Files: twisted_master.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-370 Creator: Brian Warner fix up the Twisted ProcessDocs buildstep, add TryScheduler * buildbot/process/step_twisted.py (ProcessDocs.createSummary): when creating the list of warning messages, include the line immediately after each WARNING: line, since that's usually where the file and line number wind up. * docs/examples/twisted_master.cfg: OS-X slave now does QT, add a TryScheduler --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: twisted_master.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/examples/twisted_master.cfg,v retrieving revision 1.34 retrieving revision 1.35 diff -u -d -r1.34 -r1.35 --- twisted_master.cfg 24 Oct 2005 03:48:17 -0000 1.34 +++ twisted_master.cfg 24 Oct 2005 03:48:31 -0000 1.35 @@ -13,7 +13,7 @@ from buildbot import master from buildbot.changes.pb import PBChangeSource -from buildbot.scheduler import Scheduler +from buildbot.scheduler import Scheduler, Try_Userpass from buildbot.process import step from buildbot.process.factory import s from buildbot.process.process_twisted import \ @@ -53,17 +53,6 @@ source = PBChangeSource(prefix="trunk") c['sources'].append(source) -## configure the schedulers -s_quick = Scheduler(name="quick", branch=None, treeStableTimer=30, - builderNames=["quick"]) -s_all = Scheduler(name="all", branch=None, treeStableTimer=5*60, - builderNames=["full-2.2", "full-2.3", "full-2.4", - "reactors", "OS-X", "win32", - #"freebsd", - #"threadless", - ]) -c['schedulers'] = [s_quick, s_all] - ## configure the builders @@ -200,6 +189,22 @@ c['builders'] = builders +# now set up the schedulers. We do this after setting up c['builders'] so we +# can auto-generate a list of all of them. +all_builders = [b['name'] for b in c['builders']] +all_builders.sort() + +## configure the schedulers +s_quick = Scheduler(name="quick", branch=None, treeStableTimer=30, + builderNames=["quick"]) +s_all = Scheduler(name="all", branch=None, treeStableTimer=5*60, + builderNames=all_builders) +s_try = Try_Userpass("try", all_builders, port=9988, + userpass=private.try_users) + +c['schedulers'] = [s_quick, s_all, s_try] + + # configure other status things From warner at users.sourceforge.net Mon Oct 24 04:13:08 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 04:13:08 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step_twisted.py,1.69,1.70 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv538/buildbot/process Modified Files: step_twisted.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-372 Creator: Brian Warner oops, didn't finish editing ProcessDocs.createSummary --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: step_twisted.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step_twisted.py,v retrieving revision 1.69 retrieving revision 1.70 diff -u -d -r1.69 -r1.70 --- step_twisted.py 24 Oct 2005 03:48:31 -0000 1.69 +++ step_twisted.py 24 Oct 2005 04:13:06 -0000 1.70 @@ -630,7 +630,6 @@ if wantThis: warningLines.append(line) - warningLines = filter(lambda line: line.find("WARNING: ") == 0, lines) if warningLines: self.addCompleteLog("warnings", "".join(warningLines)) self.warnings = len(warningLines) From warner at users.sourceforge.net Mon Oct 24 04:20:49 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 04:20:49 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step_twisted.py,1.70,1.71 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1925/buildbot/process Modified Files: step_twisted.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-374 Creator: Brian Warner try fixing ProcessDocs.createSummary again --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: step_twisted.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step_twisted.py,v retrieving revision 1.70 retrieving revision 1.71 diff -u -d -r1.70 -r1.71 --- step_twisted.py 24 Oct 2005 04:13:06 -0000 1.70 +++ step_twisted.py 24 Oct 2005 04:20:47 -0000 1.71 @@ -631,7 +631,7 @@ warningLines.append(line) if warningLines: - self.addCompleteLog("warnings", "".join(warningLines)) + self.addCompleteLog("warnings", "\n".join(warningLines) + "\n") self.warnings = len(warningLines) def evaluateCommand(self, cmd): From warner at users.sourceforge.net Mon Oct 24 21:39:54 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 21:39:54 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.72,1.73 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19252/buildbot/process Modified Files: step.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-376 Creator: Brian Warner improve VC handling of old buildslaves, sometimes refuse to build * buildbot/process/step.py (CVS.startVC): refuse to build update/copy -style builds on a non-default branch with an old buildslave (<=0.6.6) that doesn't know how to do it properly. The concern is that it will do a VC 'update' in an existing tree when it is supposed to be switching branches (and therefore clobbering the tree to do a full checkout), thus building the wrong source. This used to be a warning, but I think the confusion it is likely to cause warrants making it an error. (SVN.startVC): same, also make mode=export on old slaves an error (Darcs.startVC): same (Git.startVC): improve error message for non-Git-enabled slaves (Arch.checkSlaveVersion): same. continue to emit a warning when a specific revision is built on a slave that doesn't pay attention to args['revision'], because for slowly-changing trees it will probably do the right thing, and because we have no way to tell whether we're asking it to build the most recent version or not. * buildbot/interfaces.py (BuildSlaveTooOldError): new exception * buildbot/scripts/runner.py (SlaveOptions.postOptions): assert that 'master' is in host:portnum format, to catch errors sooner Index: step.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v retrieving revision 1.72 retrieving revision 1.73 diff -u -d -r1.72 -r1.73 --- step.py 22 Oct 2005 23:22:10 -0000 1.72 +++ step.py 24 Oct 2005 21:39:52 -0000 1.73 @@ -9,6 +9,7 @@ from twisted.python.failure import Failure from twisted.web.util import formatFailure +from buildbot.interfaces import BuildSlaveTooOldError from buildbot.util import now from buildbot.status import progress, builder from buildbot.status.builder import SUCCESS, WARNINGS, FAILURE, SKIPPED, \ @@ -920,13 +921,14 @@ to override __init__ and implement computeSourceRevision() and startVC(). The class as a whole builds up the self.args dictionary, then starts a LoggedRemoteCommand with those arguments. - """ # if the checkout fails, there's no point in doing anything else haltOnFailure = True notReally = False + branch = None # the default branch, should be set in __init__ + def __init__(self, workdir, mode='update', alwaysUseLatest=False, timeout=20*60, retry=None, **kwargs): """ @@ -1126,7 +1128,7 @@ it was previously performed or not. @type branch: string - @param branch: the default branch nane, will be used in a '-r' + @param branch: the default branch name, will be used in a '-r' argument to specify which branch of the source tree should be used for this checkout. Defaults to None, which means to use 'HEAD'. @@ -1189,6 +1191,23 @@ return formatdate(when) def startVC(self, branch, revision, patch): + if not self.slaveVersionNewEnough("cvs", "1.39"): + # the slave doesn't know to avoid re-using the same sourcedir + # when the branch changes. We have no way of knowing which branch + # the last build used, so if we're using a non-default branch and + # either 'update' or 'copy' modes, it is safer to refuse to + # build, and tell the user they need to upgrade the buildslave. + if (branch != self.branch + and self.args['mode'] in ("update", "copy")): + m = ("This buildslave (%s) does not know about multiple " + "branches, and using mode=%s would probably build the " + "wrong tree. " + "Refusing to build. Please upgrade the buildslave to " + "buildbot-0.7.0 or newer." % (self.build.slavename, + self.args['mode'])) + log.msg(m) + raise BuildSlaveTooOldError(m) + if branch is None: branch = "HEAD" self.args['branch'] = branch @@ -1203,19 +1222,7 @@ # deal with old slaves warnings = [] slavever = self.slaveVersion("cvs", "old") - if not self.slaveVersionNewEnough("cvs", "1.39"): - # the slave doesn't know to avoid re-using the same sourcedir - # when the branch changes. We have no way of knowing which branch - # the last build used, so if we're using a non-default branch and - # either 'update' or 'copy' modes, it is safer to switch to - # 'clobber' mode. - if branch != None and self.args['mode'] in ("update", "copy"): - m = ("buildslave does not know about multiple branches, " - "switching from mode=%s to mode=clobber to insure " - "we don't build the wrong branch." % self.args['mode']) - warnings.append(m + "\n") - log.msg(m) - self.args['mode'] = "clobber" + if slavever == "old": # 0.5.0 if self.args['mode'] == "export": @@ -1284,24 +1291,28 @@ def startVC(self, branch, revision, patch): - # accomodate old slaves + # handle old slaves warnings = [] slavever = self.slaveVersion("svn", "old") + if not slavever: + m = "slave does not have the 'svn' command" + raise BuildSlaveTooOldError(m) - assert slavever, "slave does not have the 'svn' command" if not self.slaveVersionNewEnough("svn", "1.39"): # the slave doesn't know to avoid re-using the same sourcedir # when the branch changes. We have no way of knowing which branch # the last build used, so if we're using a non-default branch and - # either 'update' or 'copy' modes, it is safer to switch to - # 'clobber' mode. - if branch != None and self.args['mode'] in ("update", "copy"): - m = ("buildslave does not know about multiple branches, " - "switching from mode=%s to mode=clobber to insure " - "we don't build the wrong branch." % self.args['mode']) - warnings.append(m + "\n") - log.msg(m) - self.args['mode'] = "clobber" + # either 'update' or 'copy' modes, it is safer to refuse to + # build, and tell the user they need to upgrade the buildslave. + if (branch != self.branch + and self.args['mode'] in ("update", "copy")): + m = ("This buildslave (%s) does not know about multiple " + "branches, and using mode=%s would probably build the " + "wrong tree. " + "Refusing to build. Please upgrade the buildslave to " + "buildbot-0.7.0 or newer." % (self.build.slavename, + self.args['mode'])) + raise BuildSlaveTooOldError(m) if slavever == "old": # 0.5.0 compatibility @@ -1312,15 +1323,22 @@ warnings.append("WARNING: this slave can only do SVN updates" ", not mode=%s\n" % self.args['mode']) log.msg("WARNING: this slave only does mode=update") - assert self.args['mode'] != "export" # more serious + if self.args['mode'] == "export": + raise BuildSlaveTooOldError("old slave does not have " + "mode=export") self.args['directory'] = self.args['workdir'] if revision is not None: - # 0.5.0 can only do HEAD - warnings.append("WARNING: this slave can only update to " - "HEAD, not revision=%s\n" % revision) - log.msg("WARNING: this slave only does -rHEAD") + # 0.5.0 can only do HEAD. We have no way of knowing whether + # the requested revision is HEAD or not, and for + # slowly-changing trees this will probably do the right + # thing, so let it pass with a warning + m = ("WARNING: old slave can only update to HEAD, not " + "revision=%s" % revision) + log.msg(m) + warnings.append(m + "\n") revision = "HEAD" # interprets this key differently - assert not patch # 0.5.0 slave can't do patch + if patch: + raise BuildSlaveTooOldError("old slave can't do patch") if self.svnurl: assert not branch # we need base_url= to use branches @@ -1379,22 +1397,30 @@ def startVC(self, branch, revision, patch): slavever = self.slaveVersion("darcs") - assert slavever, "slave is too old, does not know about darcs" + if not slavever: + m = "slave is too old, does not know about darcs" + raise BuildSlaveTooOldError(m) + if not self.slaveVersionNewEnough("darcs", "1.39"): - # 0.6.6 slaves can't handle args['revision'] - assert not revision + if revision: + # TODO: revisit this once we implement computeSourceRevision + m = "0.6.6 slaves can't handle args['revision']" + raise BuildSlaveTooOldError(m) + # the slave doesn't know to avoid re-using the same sourcedir # when the branch changes. We have no way of knowing which branch # the last build used, so if we're using a non-default branch and - # either 'update' or 'copy' modes, it is safer to switch to - # 'clobber' mode. - if branch != None and self.args['mode'] in ("update", "copy"): - m = ("buildslave does not know about multiple branches, " - "switching from mode=%s to mode=clobber to insure " - "we don't build the wrong branch." % self.args['mode']) - warnings.append(m + "\m") - log.msg(m) - self.args['mode'] = "clobber" + # either 'update' or 'copy' modes, it is safer to refuse to + # build, and tell the user they need to upgrade the buildslave. + if (branch != self.branch + and self.args['mode'] in ("update", "copy")): + m = ("This buildslave (%s) does not know about multiple " + "branches, and using mode=%s would probably build the " + "wrong tree. " + "Refusing to build. Please upgrade the buildslave to " + "buildbot-0.7.0 or newer." % (self.build.slavename, + self.args['mode'])) + raise BuildSlaveTooOldError(m) if self.repourl: assert not branch # we need base_url= to use branches @@ -1426,7 +1452,9 @@ self.args['revision'] = revision self.args['patch'] = patch slavever = self.slaveVersion("git") - assert slavever, "slave is too old, does not know about git" + if not slavever: + raise BuildSlaveTooOldError("slave is too old, does not know " + "about git") self.cmd = LoggedRemoteCommand("git", self.args) ShellCommand.start(self) @@ -1489,36 +1517,48 @@ return "base-0" return "patch-%d" % lastChange - def checkSlaveVersion(self, cmd): + def checkSlaveVersion(self, cmd, branch): warnings = [] slavever = self.slaveVersion(cmd) - assert slavever, "slave is too old, does not know about arch" + if not slavever: + m = "slave is too old, does not know about %s" % cmd + raise BuildSlaveTooOldError(m) + # slave 1.28 and later understand 'revision' if not self.slaveVersionNewEnough(cmd, "1.28"): if not self.alwaysUseLatest: - log.msg("warning, buildslave is too old to use a revision") - warnings.append("buildslave is too old to use a revision") + # we don't know whether our requested revision is the latest + # or not. If the tree does not change very quickly, this will + # probably build the right thing, so emit a warning rather + # than refuse to build at all + m = "WARNING, buildslave is too old to use a revision" + log.msg(m) + warnings.append(m + "\n") if not self.slaveVersionNewEnough(cmd, "1.39"): # the slave doesn't know to avoid re-using the same sourcedir # when the branch changes. We have no way of knowing which branch # the last build used, so if we're using a non-default branch and - # either 'update' or 'copy' modes, it is safer to switch to - # 'clobber' mode. - if branch != None and self.args['mode'] in ("update", "copy"): - m = ("buildslave does not know about multiple branches, " - "switching from mode=%s to mode=clobber to insure " - "we don't build the wrong branch." % self.args['mode']) - warnings.append(m + "\n") + # either 'update' or 'copy' modes, it is safer to refuse to + # build, and tell the user they need to upgrade the buildslave. + if (branch != self.branch + and self.args['mode'] in ("update", "copy")): + m = ("This buildslave (%s) does not know about multiple " + "branches, and using mode=%s would probably build the " + "wrong tree. " + "Refusing to build. Please upgrade the buildslave to " + "buildbot-0.7.0 or newer." % (self.build.slavename, + self.args['mode'])) log.msg(m) - self.args['mode'] = "clobber" + raise BuildslaveError(m) + return warnings def startVC(self, branch, revision, patch): self.args['version'] = branch self.args['revision'] = revision self.args['patch'] = patch - warnings = self.checkSlaveVersion("arch") + warnings = self.checkSlaveVersion("arch", branch) self.cmd = LoggedRemoteCommand("arch", self.args) ShellCommand.start(self, warnings) @@ -1555,7 +1595,7 @@ self.args['version'] = branch self.args['revision'] = revision self.args['patch'] = patch - warnings = self.checkSlaveVersion("bazaar") + warnings = self.checkSlaveVersion("bazaar", branch) self.cmd = LoggedRemoteCommand("bazaar", self.args) ShellCommand.start(self, warnings) From warner at users.sourceforge.net Mon Oct 24 21:39:54 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 21:39:54 +0000 Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.36,1.37 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19252/buildbot Modified Files: interfaces.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-376 Creator: Brian Warner improve VC handling of old buildslaves, sometimes refuse to build * buildbot/process/step.py (CVS.startVC): refuse to build update/copy -style builds on a non-default branch with an old buildslave (<=0.6.6) that doesn't know how to do it properly. The concern is that it will do a VC 'update' in an existing tree when it is supposed to be switching branches (and therefore clobbering the tree to do a full checkout), thus building the wrong source. This used to be a warning, but I think the confusion it is likely to cause warrants making it an error. (SVN.startVC): same, also make mode=export on old slaves an error (Darcs.startVC): same (Git.startVC): improve error message for non-Git-enabled slaves (Arch.checkSlaveVersion): same. continue to emit a warning when a specific revision is built on a slave that doesn't pay attention to args['revision'], because for slowly-changing trees it will probably do the right thing, and because we have no way to tell whether we're asking it to build the most recent version or not. * buildbot/interfaces.py (BuildSlaveTooOldError): new exception * buildbot/scripts/runner.py (SlaveOptions.postOptions): assert that 'master' is in host:portnum format, to catch errors sooner Index: interfaces.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v retrieving revision 1.36 retrieving revision 1.37 diff -u -d -r1.36 -r1.37 --- interfaces.py 21 Oct 2005 08:03:39 -0000 1.36 +++ interfaces.py 24 Oct 2005 21:39:52 -0000 1.37 @@ -12,6 +12,8 @@ pass class BuilderInUseError(Exception): pass +class BuildSlaveTooOldError(Exception): + pass class IChangeSource(Interface): """Object which feeds Change objects to the changemaster. When files or From warner at users.sourceforge.net Mon Oct 24 21:39:54 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 21:39:54 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.530,1.531 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19252 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-376 Creator: Brian Warner improve VC handling of old buildslaves, sometimes refuse to build * buildbot/process/step.py (CVS.startVC): refuse to build update/copy -style builds on a non-default branch with an old buildslave (<=0.6.6) that doesn't know how to do it properly. The concern is that it will do a VC 'update' in an existing tree when it is supposed to be switching branches (and therefore clobbering the tree to do a full checkout), thus building the wrong source. This used to be a warning, but I think the confusion it is likely to cause warrants making it an error. (SVN.startVC): same, also make mode=export on old slaves an error (Darcs.startVC): same (Git.startVC): improve error message for non-Git-enabled slaves (Arch.checkSlaveVersion): same. continue to emit a warning when a specific revision is built on a slave that doesn't pay attention to args['revision'], because for slowly-changing trees it will probably do the right thing, and because we have no way to tell whether we're asking it to build the most recent version or not. * buildbot/interfaces.py (BuildSlaveTooOldError): new exception * buildbot/scripts/runner.py (SlaveOptions.postOptions): assert that 'master' is in host:portnum format, to catch errors sooner Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.530 retrieving revision 1.531 diff -u -d -r1.530 -r1.531 --- ChangeLog 24 Oct 2005 03:48:30 -0000 1.530 +++ ChangeLog 24 Oct 2005 21:39:52 -0000 1.531 @@ -1,3 +1,26 @@ +2005-10-24 Brian Warner + + * buildbot/process/step.py (CVS.startVC): refuse to build + update/copy -style builds on a non-default branch with an old + buildslave (<=0.6.6) that doesn't know how to do it properly. The + concern is that it will do a VC 'update' in an existing tree when + it is supposed to be switching branches (and therefore clobbering + the tree to do a full checkout), thus building the wrong source. + This used to be a warning, but I think the confusion it is likely + to cause warrants making it an error. + (SVN.startVC): same, also make mode=export on old slaves an error + (Darcs.startVC): same + (Git.startVC): improve error message for non-Git-enabled slaves + (Arch.checkSlaveVersion): same. continue to emit a warning when a + specific revision is built on a slave that doesn't pay attention + to args['revision'], because for slowly-changing trees it will + probably do the right thing, and because we have no way to tell + whether we're asking it to build the most recent version or not. + * buildbot/interfaces.py (BuildSlaveTooOldError): new exception + + * buildbot/scripts/runner.py (SlaveOptions.postOptions): assert + that 'master' is in host:portnum format, to catch errors sooner + 2005-10-23 Brian Warner * buildbot/process/step_twisted.py (ProcessDocs.createSummary): From warner at users.sourceforge.net Mon Oct 24 21:39:54 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 21:39:54 +0000 Subject: [Buildbot-commits] buildbot/buildbot/scripts runner.py,1.37,1.38 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/scripts In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19252/buildbot/scripts Modified Files: runner.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-376 Creator: Brian Warner improve VC handling of old buildslaves, sometimes refuse to build * buildbot/process/step.py (CVS.startVC): refuse to build update/copy -style builds on a non-default branch with an old buildslave (<=0.6.6) that doesn't know how to do it properly. The concern is that it will do a VC 'update' in an existing tree when it is supposed to be switching branches (and therefore clobbering the tree to do a full checkout), thus building the wrong source. This used to be a warning, but I think the confusion it is likely to cause warrants making it an error. (SVN.startVC): same, also make mode=export on old slaves an error (Darcs.startVC): same (Git.startVC): improve error message for non-Git-enabled slaves (Arch.checkSlaveVersion): same. continue to emit a warning when a specific revision is built on a slave that doesn't pay attention to args['revision'], because for slowly-changing trees it will probably do the right thing, and because we have no way to tell whether we're asking it to build the most recent version or not. * buildbot/interfaces.py (BuildSlaveTooOldError): new exception * buildbot/scripts/runner.py (SlaveOptions.postOptions): assert that 'master' is in host:portnum format, to catch errors sooner Index: runner.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/runner.py,v retrieving revision 1.37 retrieving revision 1.38 diff -u -d -r1.37 -r1.38 --- runner.py 21 Oct 2005 08:03:39 -0000 1.37 +++ runner.py 24 Oct 2005 21:39:52 -0000 1.38 @@ -258,6 +258,8 @@ MakerBase.postOptions(self) self['usepty'] = int(self['usepty']) self['keepalive'] = int(self['keepalive']) + if self['master'].find(":") == -1: + raise usage.UsageError("--master must be in the form host:portnum") slaveTAC = """ from twisted.application import service From warner at users.sourceforge.net Mon Oct 24 21:49:21 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 21:49:21 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.531,1.532 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21603 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-378 Creator: Brian Warner s/slaveVersionNewEnough/slaveVersionIsOlderThan/ * buildbot/process/step.py (BuildStep): rename slaveVersionNewEnough to slaveVersionIsOlderThan, because that's how it is normally used. * buildbot/test/test_steps.py (Version.checkCompare): same Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.531 retrieving revision 1.532 diff -u -d -r1.531 -r1.532 --- ChangeLog 24 Oct 2005 21:39:52 -0000 1.531 +++ ChangeLog 24 Oct 2005 21:49:19 -0000 1.532 @@ -1,5 +1,10 @@ 2005-10-24 Brian Warner + * buildbot/process/step.py (BuildStep): rename + slaveVersionNewEnough to slaveVersionIsOlderThan, because that's + how it is normally used. + * buildbot/test/test_steps.py (Version.checkCompare): same + * buildbot/process/step.py (CVS.startVC): refuse to build update/copy -style builds on a non-default branch with an old buildslave (<=0.6.6) that doesn't know how to do it properly. The From warner at users.sourceforge.net Mon Oct 24 21:49:20 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 21:49:20 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.73,1.74 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21603/buildbot/process Modified Files: step.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-378 Creator: Brian Warner s/slaveVersionNewEnough/slaveVersionIsOlderThan/ * buildbot/process/step.py (BuildStep): rename slaveVersionNewEnough to slaveVersionIsOlderThan, because that's how it is normally used. * buildbot/test/test_steps.py (Version.checkCompare): same Index: step.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v retrieving revision 1.73 retrieving revision 1.74 diff -u -d -r1.73 -r1.74 --- step.py 24 Oct 2005 21:39:52 -0000 1.73 +++ step.py 24 Oct 2005 21:49:18 -0000 1.74 @@ -632,16 +632,16 @@ """ return self.build.getSlaveCommandVersion(command, oldversion) - def slaveVersionNewEnough(self, command, minversion): + def slaveVersionIsOlderThan(self, command, minversion): sv = self.build.getSlaveCommandVersion(command, None) if sv is None: - return False + return True # the version we get back is a string form of the CVS version number # of the slave's buildbot/slave/commands.py, something like 1.39 . # This might change in the future (I might move away from CVS), but # if so I'll keep updating that string with suitably-comparable # values. - if sv.split(".") >= minversion.split("."): + if sv.split(".") < minversion.split("."): return True return False @@ -1191,7 +1191,7 @@ return formatdate(when) def startVC(self, branch, revision, patch): - if not self.slaveVersionNewEnough("cvs", "1.39"): + if self.slaveVersionIsOlderThan("cvs", "1.39"): # the slave doesn't know to avoid re-using the same sourcedir # when the branch changes. We have no way of knowing which branch # the last build used, so if we're using a non-default branch and @@ -1298,7 +1298,7 @@ m = "slave does not have the 'svn' command" raise BuildSlaveTooOldError(m) - if not self.slaveVersionNewEnough("svn", "1.39"): + if self.slaveVersionIsOlderThan("svn", "1.39"): # the slave doesn't know to avoid re-using the same sourcedir # when the branch changes. We have no way of knowing which branch # the last build used, so if we're using a non-default branch and @@ -1401,7 +1401,7 @@ m = "slave is too old, does not know about darcs" raise BuildSlaveTooOldError(m) - if not self.slaveVersionNewEnough("darcs", "1.39"): + if self.slaveVersionIsOlderThan("darcs", "1.39"): if revision: # TODO: revisit this once we implement computeSourceRevision m = "0.6.6 slaves can't handle args['revision']" @@ -1525,7 +1525,7 @@ raise BuildSlaveTooOldError(m) # slave 1.28 and later understand 'revision' - if not self.slaveVersionNewEnough(cmd, "1.28"): + if self.slaveVersionIsOlderThan(cmd, "1.28"): if not self.alwaysUseLatest: # we don't know whether our requested revision is the latest # or not. If the tree does not change very quickly, this will @@ -1535,7 +1535,7 @@ log.msg(m) warnings.append(m + "\n") - if not self.slaveVersionNewEnough(cmd, "1.39"): + if self.slaveVersionIsOlderThan(cmd, "1.39"): # the slave doesn't know to avoid re-using the same sourcedir # when the branch changes. We have no way of knowing which branch # the last build used, so if we're using a non-default branch and From warner at users.sourceforge.net Mon Oct 24 21:49:21 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 21:49:21 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_steps.py,1.16,1.17 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21603/buildbot/test Modified Files: test_steps.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-378 Creator: Brian Warner s/slaveVersionNewEnough/slaveVersionIsOlderThan/ * buildbot/process/step.py (BuildStep): rename slaveVersionNewEnough to slaveVersionIsOlderThan, because that's how it is normally used. * buildbot/test/test_steps.py (Version.checkCompare): same Index: test_steps.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- test_steps.py 23 Oct 2005 05:28:46 -0000 1.16 +++ test_steps.py 24 Oct 2005 21:49:18 -0000 1.17 @@ -224,9 +224,10 @@ #self.failUnlessEqual(s.slaveVersion("NOSUCHCOMMAND", "old"), "old") # now check the comparison functions - self.failUnless(s.slaveVersionNewEnough("svn", commands.cvs_ver)) - self.failUnless(s.slaveVersionNewEnough("svn", "1.1")) - self.failIf(s.slaveVersionNewEnough("svn", commands.cvs_ver + ".1")) + self.failIf(s.slaveVersionIsOlderThan("svn", commands.cvs_ver)) + self.failIf(s.slaveVersionIsOlderThan("svn", "1.1")) + self.failUnless(s.slaveVersionIsOlderThan("svn", + commands.cvs_ver + ".1")) def testCompare(self): self.master._checker = self.checkCompare From warner at users.sourceforge.net Mon Oct 24 22:42:05 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 22:42:05 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.532,1.533 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4679 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-380 Creator: Brian Warner s/base_url/baseURL/ and s/default_branch/defaultBranch/ for SVN+Darcs * buildbot/process/step.py (SVN.__init__): change 'base_url' and 'default_branch' argument names to 'baseURL' and 'defaultBranch', for consistency with other BuildStep arguments that use camelCase. Well, at least more of them use camelCase (like flunkOnWarnings) than don't.. I wish I'd picked one style and stuck with it earlier. Annoying, but it's best done before the release, since these arguments didn't exist at all in 0.6.6 . (Darcs): same * buildbot/test/test_vc.py (SVN.testCheckout): same (Darcs.testPatch): same * docs/buildbot.texinfo (SVN): document the change (Darcs): same, add some build-on-branch docs * docs/examples/twisted_master.cfg: match change Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.532 retrieving revision 1.533 diff -u -d -r1.532 -r1.533 --- ChangeLog 24 Oct 2005 21:49:19 -0000 1.532 +++ ChangeLog 24 Oct 2005 22:42:03 -0000 1.533 @@ -1,5 +1,19 @@ 2005-10-24 Brian Warner + * buildbot/process/step.py (SVN.__init__): change 'base_url' and + 'default_branch' argument names to 'baseURL' and 'defaultBranch', + for consistency with other BuildStep arguments that use camelCase. + Well, at least more of them use camelCase (like flunkOnWarnings) + than don't.. I wish I'd picked one style and stuck with it + earlier. Annoying, but it's best done before the release, since + these arguments didn't exist at all in 0.6.6 . + (Darcs): same + * buildbot/test/test_vc.py (SVN.testCheckout): same + (Darcs.testPatch): same + * docs/buildbot.texinfo (SVN): document the change + (Darcs): same, add some build-on-branch docs + * docs/examples/twisted_master.cfg: match change + * buildbot/process/step.py (BuildStep): rename slaveVersionNewEnough to slaveVersionIsOlderThan, because that's how it is normally used. From warner at users.sourceforge.net Mon Oct 24 22:42:05 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 22:42:05 +0000 Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.74,1.75 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/process In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4679/buildbot/process Modified Files: step.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-380 Creator: Brian Warner s/base_url/baseURL/ and s/default_branch/defaultBranch/ for SVN+Darcs * buildbot/process/step.py (SVN.__init__): change 'base_url' and 'default_branch' argument names to 'baseURL' and 'defaultBranch', for consistency with other BuildStep arguments that use camelCase. Well, at least more of them use camelCase (like flunkOnWarnings) than don't.. I wish I'd picked one style and stuck with it earlier. Annoying, but it's best done before the release, since these arguments didn't exist at all in 0.6.6 . (Darcs): same * buildbot/test/test_vc.py (SVN.testCheckout): same (Darcs.testPatch): same * docs/buildbot.texinfo (SVN): document the change (Darcs): same, add some build-on-branch docs * docs/examples/twisted_master.cfg: match change Index: step.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v retrieving revision 1.74 retrieving revision 1.75 diff -u -d -r1.74 -r1.75 --- step.py 24 Oct 2005 21:49:18 -0000 1.74 +++ step.py 24 Oct 2005 22:42:03 -0000 1.75 @@ -1243,7 +1243,7 @@ name = 'svn' - def __init__(self, svnurl=None, base_url=None, default_branch=None, + def __init__(self, svnurl=None, baseURL=None, defaultBranch=None, directory=None, **kwargs): """ @type svnurl: string @@ -1252,19 +1252,19 @@ the repository host/port, the repository path, the sub-tree within the repository, and the branch to check out. Using C{svnurl} does not enable builds of - alternate branches: use C{base_url} to enable this. - Use exactly one of C{svnurl} and C{base_url}. + alternate branches: use C{baseURL} to enable this. + Use exactly one of C{svnurl} and C{baseURL}. - @param base_url: if branches are enabled, this is the base URL to - which a branch name will be appended. It should - probably end in a slash. Use exactly one of - C{svnurl} and C{base_url}. + @param baseURL: if branches are enabled, this is the base URL to + which a branch name will be appended. It should + probably end in a slash. Use exactly one of + C{svnurl} and C{baseURL}. - @param default_branch: if branches are enabled, this is the branch - to use if the Build does not specify one - explicitly. It will simply be appended - to C{base_url} and the result handed to - the SVN command. + @param defaultBranch: if branches are enabled, this is the branch + to use if the Build does not specify one + explicitly. It will simply be appended + to C{baseURL} and the result handed to + the SVN command. """ if not kwargs.has_key('workdir') and directory is not None: @@ -1273,12 +1273,12 @@ DeprecationWarning) kwargs['workdir'] = directory - if not svnurl and not base_url: - raise ValueError("you must use exactly one of svnurl and base_url") + if not svnurl and not baseURL: + raise ValueError("you must use exactly one of svnurl and baseURL") self.svnurl = svnurl - self.base_url = base_url - self.branch = default_branch + self.baseURL = baseURL + self.branch = defaultBranch Source.__init__(self, **kwargs) @@ -1341,10 +1341,10 @@ raise BuildSlaveTooOldError("old slave can't do patch") if self.svnurl: - assert not branch # we need base_url= to use branches + assert not branch # we need baseURL= to use branches self.args['svnurl'] = self.svnurl else: - self.args['svnurl'] = self.base_url + branch + self.args['svnurl'] = self.baseURL + branch self.args['revision'] = revision self.args['patch'] = patch @@ -1364,35 +1364,35 @@ name = "darcs" - def __init__(self, repourl=None, base_url=None, default_branch=None, + def __init__(self, repourl=None, baseURL=None, defaultBranch=None, **kwargs): """ @type repourl: string @param repourl: the URL which points at the Darcs repository. This is used as the default branch. Using C{repourl} does not enable builds of alternate branches: use - C{base_url} to enable this. Use either C{repourl} or - C{base_url}, not both. + C{baseURL} to enable this. Use either C{repourl} or + C{baseURL}, not both. - @param base_url: if branches are enabled, this is the base URL to - which a branch name will be appended. It should - probably end in a slash. Use exactly one of - C{repourl} and C{base_url}. + @param baseURL: if branches are enabled, this is the base URL to + which a branch name will be appended. It should + probably end in a slash. Use exactly one of + C{repourl} and C{baseURL}. - @param default_branch: if branches are enabled, this is the branch - to use if the Build does not specify one - explicitly. It will simply be appended to - C{base_url} and the result handed to the - 'darcs pull' command. + @param defaultBranch: if branches are enabled, this is the branch + to use if the Build does not specify one + explicitly. It will simply be appended to + C{baseURL} and the result handed to the + 'darcs pull' command. """ assert kwargs['mode'] != "export", \ "Darcs does not have an 'export' mode" - if (not repourl and not base_url) or (repourl and base_url): + if (not repourl and not baseURL) or (repourl and baseURL): raise ValueError("you must provide exactly one of repourl and" - " base_url") + " baseURL") self.repourl = repourl - self.base_url = base_url - self.branch = default_branch + self.baseURL = baseURL + self.branch = defaultBranch Source.__init__(self, **kwargs) def startVC(self, branch, revision, patch): @@ -1423,10 +1423,10 @@ raise BuildSlaveTooOldError(m) if self.repourl: - assert not branch # we need base_url= to use branches + assert not branch # we need baseURL= to use branches self.args['repourl'] = self.repourl else: - self.args['repourl'] = self.base_url + branch + self.args['repourl'] = self.baseURL + branch self.args['revision'] = revision self.args['patch'] = patch self.cmd = LoggedRemoteCommand("darcs", self.args) From warner at users.sourceforge.net Mon Oct 24 22:42:05 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 22:42:05 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_vc.py,1.42,1.43 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4679/buildbot/test Modified Files: test_vc.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-380 Creator: Brian Warner s/base_url/baseURL/ and s/default_branch/defaultBranch/ for SVN+Darcs * buildbot/process/step.py (SVN.__init__): change 'base_url' and 'default_branch' argument names to 'baseURL' and 'defaultBranch', for consistency with other BuildStep arguments that use camelCase. Well, at least more of them use camelCase (like flunkOnWarnings) than don't.. I wish I'd picked one style and stuck with it earlier. Annoying, but it's best done before the release, since these arguments didn't exist at all in 0.6.6 . (Darcs): same * buildbot/test/test_vc.py (SVN.testCheckout): same (Darcs.testPatch): same * docs/buildbot.texinfo (SVN): document the change (Darcs): same, add some build-on-branch docs * docs/examples/twisted_master.cfg: match change Index: test_vc.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v retrieving revision 1.42 retrieving revision 1.43 diff -u -d -r1.42 -r1.43 --- test_vc.py 16 Aug 2005 00:23:53 -0000 1.42 +++ test_vc.py 24 Oct 2005 22:42:03 -0000 1.43 @@ -1108,21 +1108,21 @@ def testCheckout(self): # we verify this one with the svnurl style of vcargs. We test the - # base_url/default_branch style in testPatch and testBranch. + # baseURL/defaultBranch style in testPatch and testBranch. self.vcargs = { 'svnurl': self.svnurl_trunk } d = self.do_vc() return maybeWait(d) def testPatch(self): - self.vcargs = { 'base_url': self.svnurl + "/", - 'default_branch': "sample/trunk", + self.vcargs = { 'baseURL': self.svnurl + "/", + 'defaultBranch': "sample/trunk", } d = self.do_patch() return maybeWait(d) def testBranch(self): - self.vcargs = { 'base_url': self.svnurl + "/", - 'default_branch': "sample/trunk", + self.vcargs = { 'baseURL': self.svnurl + "/", + 'defaultBranch': "sample/trunk", } d = self.do_branch() return maybeWait(d) @@ -1130,8 +1130,8 @@ def testTry(self): # extract the base revision and patch from a modified tree, use it to # create the same contents on the buildslave - self.vcargs = { 'base_url': self.svnurl + "/", - 'default_branch': "sample/trunk", + self.vcargs = { 'baseURL': self.svnurl + "/", + 'defaultBranch': "sample/trunk", } d = self.do_getpatch() return maybeWait(d) @@ -1238,14 +1238,14 @@ return maybeWait(d) def testPatch(self): - self.vcargs = { 'base_url': self.darcs_base + "/", - 'default_branch': "trunk" } + self.vcargs = { 'baseURL': self.darcs_base + "/", + 'defaultBranch': "trunk" } d = self.do_patch() return maybeWait(d) def testBranch(self): - self.vcargs = { 'base_url': self.darcs_base + "/", - 'default_branch': "trunk" } + self.vcargs = { 'baseURL': self.darcs_base + "/", + 'defaultBranch': "trunk" } d = self.do_branch() return maybeWait(d) @@ -1259,8 +1259,8 @@ return maybeWait(d) def testTry(self): - self.vcargs = { 'base_url': self.darcs_base + "/", - 'default_branch': "trunk" } + self.vcargs = { 'baseURL': self.darcs_base + "/", + 'defaultBranch': "trunk" } d = self.do_getpatch() return maybeWait(d) From warner at users.sourceforge.net Mon Oct 24 22:42:05 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 22:42:05 +0000 Subject: [Buildbot-commits] buildbot/docs/examples twisted_master.cfg,1.35,1.36 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/examples In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4679/docs/examples Modified Files: twisted_master.cfg Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-380 Creator: Brian Warner s/base_url/baseURL/ and s/default_branch/defaultBranch/ for SVN+Darcs * buildbot/process/step.py (SVN.__init__): change 'base_url' and 'default_branch' argument names to 'baseURL' and 'defaultBranch', for consistency with other BuildStep arguments that use camelCase. Well, at least more of them use camelCase (like flunkOnWarnings) than don't.. I wish I'd picked one style and stuck with it earlier. Annoying, but it's best done before the release, since these arguments didn't exist at all in 0.6.6 . (Darcs): same * buildbot/test/test_vc.py (SVN.testCheckout): same (Darcs.testPatch): same * docs/buildbot.texinfo (SVN): document the change (Darcs): same, add some build-on-branch docs * docs/examples/twisted_master.cfg: match change Index: twisted_master.cfg =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/examples/twisted_master.cfg,v retrieving revision 1.35 retrieving revision 1.36 diff -u -d -r1.35 -r1.36 --- twisted_master.cfg 24 Oct 2005 03:48:31 -0000 1.35 +++ twisted_master.cfg 24 Oct 2005 22:42:03 -0000 1.36 @@ -56,19 +56,22 @@ ## configure the builders -svnurl = "svn://svn.twistedmatrix.com/svn/Twisted/trunk" -# for build-on-branch, we use these instead -base_url = "svn://svn.twistedmatrix.com/svn/Twisted/" -default_branch = "trunk" -#source_update = s(step.SVN, svnurl=svnurl, mode="update") -#source_copy = s(step.SVN, svnurl=svnurl, mode="copy") -#source_export = s(step.SVN, svnurl=svnurl, mode="export") -source_update = s(step.SVN, base_url=base_url, default_branch=default_branch, - mode="update") -source_copy = s(step.SVN, base_url=base_url, default_branch=default_branch, - mode="copy") -source_export = s(step.SVN, base_url=base_url, default_branch=default_branch, - mode="export") +if 0: + # always build on trunk + svnurl = "svn://svn.twistedmatrix.com/svn/Twisted/trunk" + source_update = s(step.SVN, svnurl=svnurl, mode="update") + source_copy = s(step.SVN, svnurl=svnurl, mode="copy") + source_export = s(step.SVN, svnurl=svnurl, mode="export") +else: + # for build-on-branch, we use these instead + baseURL = "svn://svn.twistedmatrix.com/svn/Twisted/" + defaultBranch = "trunk" + source_update = s(step.SVN, baseURL=baseURL, defaultBranch=defaultBranch, + mode="update") + source_copy = s(step.SVN, baseURL=baseURL, defaultBranch=defaultBranch, + mode="copy") + source_export = s(step.SVN, baseURL=baseURL, defaultBranch=defaultBranch, + mode="export") builders = [] From warner at users.sourceforge.net Mon Oct 24 22:42:05 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Mon, 24 Oct 2005 22:42:05 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.23,1.24 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4679/docs Modified Files: buildbot.texinfo Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-380 Creator: Brian Warner s/base_url/baseURL/ and s/default_branch/defaultBranch/ for SVN+Darcs * buildbot/process/step.py (SVN.__init__): change 'base_url' and 'default_branch' argument names to 'baseURL' and 'defaultBranch', for consistency with other BuildStep arguments that use camelCase. Well, at least more of them use camelCase (like flunkOnWarnings) than don't.. I wish I'd picked one style and stuck with it earlier. Annoying, but it's best done before the release, since these arguments didn't exist at all in 0.6.6 . (Darcs): same * buildbot/test/test_vc.py (SVN.testCheckout): same (Darcs.testPatch): same * docs/buildbot.texinfo (SVN): document the change (Darcs): same, add some build-on-branch docs * docs/examples/twisted_master.cfg: match change Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- buildbot.texinfo 14 Oct 2005 19:42:40 -0000 1.23 +++ buildbot.texinfo 24 Oct 2005 22:42:03 -0000 1.24 @@ -2396,8 +2396,8 @@ @cindex CVS Checkout -The @code{CVS} build step performs a CVS checkout or update. It -takes the following arguments: +The @code{CVS} build step performs a @uref{http://www.nongnu.org/cvs/, +CVS} checkout or update. It takes the following arguments: @table @code @item cvsroot @@ -2431,7 +2431,8 @@ @cindex SVN Checkout -The @code{SVN} build step performs a Subversion checkout or update. +The @code{SVN} build step performs a + at uref{http://subversion.tigris.org, Subversion} checkout or update. There are two basic ways of setting up the checkout step, depending upon whether you are using branches or not. @@ -2453,17 +2454,17 @@ @end table If, on the other hand, you are building from multiple branches, then -you should create the @code{SVN} step with the @code{base_url} and - at code{default_branch} arguments instead: +you should create the @code{SVN} step with the @code{baseURL} and + at code{defaultBranch} arguments instead: @table @code - at item base_url + at item baseURL (required): this specifies the base repository URL, to which a branch name will be appended. It should probably end in a slash. - at item default_branch + at item defaultBranch this specifies the name of the branch to use when a Build does not -provide one of its own. This will be appended to @code{base_url} to +provide one of its own. This will be appended to @code{baseURL} to create the string that will be passed to the @code{svn checkout} command. @end table @@ -2522,7 +2523,7 @@ it to pay attention to all branches. The third piece is an @code{SVN} checkout step that is configured to -handle the branches correctly, with a @code{base_url} value that +handle the branches correctly, with a @code{baseURL} value that matches the way the ChangeSource splits each file's URL into base, branch, and file. @@ -2538,8 +2539,8 @@ 10*60, ['test-i386', 'test-ppc']) c['schedulers'] = [s] source = s(step.SVN, mode='update', - base_url='svn://svn.example.org/MyProject/', - default_branch='trunk') + baseURL='svn://svn.example.org/MyProject/', + defaultBranch='trunk') f = factory.BuildFactory([source, s(step.Compile, command="make all"), s(step.Test, command="make test")]) @@ -2553,7 +2554,7 @@ In this example, when a change arrives with a @code{branch} attribute of ``trunk'', the resulting build will have an SVN step that -concatenates ``svn://svn.example.org/MyProject/'' (the base_url) with +concatenates ``svn://svn.example.org/MyProject/'' (the baseURL) with ``trunk'' (the branch name) to get the correct svn command. If the ``newthing'' branch has a change to ``src/foo.c'', then the SVN step will concatenate ``svn://svn.example.org/MyProject/'' with @@ -2565,12 +2566,40 @@ @cindex Darcs Checkout The @code{Darcs} build step performs a - at uref{http://abridgegame.org/darcs/, Darcs} checkout or update. It -takes the following arguments: + at uref{http://abridgegame.org/darcs/, Darcs} checkout or update. + +Like @xref{SVN}, this step can either be configured to always check +out a specific tree, or set up to pull from a particular branch that +gets specified separately for each build. Also like SVN, the +repository URL given to Darcs is created by concatenating a + at code{baseURL} with the branch name, and if no particular branch is +requested, it uses a @code{defaultBranch}. The only difference in +usage is that each potential Darcs repository URL must point to a +fully-fledged repository, whereas SVN URLs usually point to sub-trees +of the main Subversion repository. In other words, doing an SVN +checkout of @code{baseURL} is legal, but silly, since you'd probably +wind up with a copy of every single branch in the whole repository. +Doing a Darcs checkout of @code{baseURL} is just plain wrong, since +the parent directory of a collection of Darcs repositories is not +itself a valid repository. + +The Darcs step takes the following arguments: @table @code @item repourl -(required): the URL at which the Darcs source repository is available. +(required unless @code{baseURL} is provided): the URL at which the +Darcs source repository is available. + + at item baseURL +(required unless @code{repourl} is provided): the base repository URL, +to which a branch name will be appended. It should probably end in a +slash. + + at item defaultBranch +(allowed only if @code{baseURL} is provided): this specifies the name +of the branch to use when a Build does not provide one of its own. +This will be appended to @code{baseURL} to create the string that will +be passed to the @code{darcs get} command. @end table @@ -2590,7 +2619,8 @@ @item version (required): this specifies which ``development line'' (like a branch) -should be used. +should be used. This provides the default branch name, but individual +builds may specify a different one. @item archive (optional): Each repository knows its own archive name. If this From warner at users.sourceforge.net Tue Oct 25 01:26:13 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:26:13 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.533,1.534 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8323 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-382 Creator: Brian Warner update manual, add --branch to 'buildbot sendchange' * docs/buildbot.texinfo: update lots of stuff * buildbot/scripts/runner.py (sendchange): add a --branch argument to the 'buildbot sendchange' command * buildbot/clients/sendchange.py (Sender.send): same * buildbot/changes/pb.py (ChangePerspective): same * buildbot/test/test_changes.py (Sender.testSender): test it Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.533 retrieving revision 1.534 diff -u -d -r1.533 -r1.534 --- ChangeLog 24 Oct 2005 22:42:03 -0000 1.533 +++ ChangeLog 25 Oct 2005 01:26:11 -0000 1.534 @@ -1,5 +1,13 @@ 2005-10-24 Brian Warner + * docs/buildbot.texinfo: update lots of stuff + + * buildbot/scripts/runner.py (sendchange): add a --branch argument + to the 'buildbot sendchange' command + * buildbot/clients/sendchange.py (Sender.send): same + * buildbot/changes/pb.py (ChangePerspective): same + * buildbot/test/test_changes.py (Sender.testSender): test it + * buildbot/process/step.py (SVN.__init__): change 'base_url' and 'default_branch' argument names to 'baseURL' and 'defaultBranch', for consistency with other BuildStep arguments that use camelCase. From warner at users.sourceforge.net Tue Oct 25 01:26:13 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:26:13 +0000 Subject: [Buildbot-commits] buildbot/buildbot/changes pb.py,1.8,1.9 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/changes In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8323/buildbot/changes Modified Files: pb.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-382 Creator: Brian Warner update manual, add --branch to 'buildbot sendchange' * docs/buildbot.texinfo: update lots of stuff * buildbot/scripts/runner.py (sendchange): add a --branch argument to the 'buildbot sendchange' command * buildbot/clients/sendchange.py (Sender.send): same * buildbot/changes/pb.py (ChangePerspective): same * buildbot/test/test_changes.py (Sender.testSender): test it Index: pb.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/changes/pb.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- pb.py 17 May 2005 10:14:09 -0000 1.8 +++ pb.py 25 Oct 2005 01:26:11 -0000 1.9 @@ -42,7 +42,9 @@ change = changes.Change(changedict['who'], pathnames, changedict['comments'], - revision=changedict.get('revision')) + branch=changedict.get('branch'), + revision=changedict.get('revision'), + ) self.changemaster.addChange(change) class PBChangeSource(base.ChangeSource): From warner at users.sourceforge.net Tue Oct 25 01:26:13 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:26:13 +0000 Subject: [Buildbot-commits] buildbot/buildbot/scripts runner.py,1.38,1.39 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/scripts In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8323/buildbot/scripts Modified Files: runner.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-382 Creator: Brian Warner update manual, add --branch to 'buildbot sendchange' * docs/buildbot.texinfo: update lots of stuff * buildbot/scripts/runner.py (sendchange): add a --branch argument to the 'buildbot sendchange' command * buildbot/clients/sendchange.py (Sender.send): same * buildbot/changes/pb.py (ChangePerspective): same * buildbot/test/test_changes.py (Sender.testSender): test it Index: runner.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/runner.py,v retrieving revision 1.38 retrieving revision 1.39 diff -u -d -r1.38 -r1.39 --- runner.py 24 Oct 2005 21:39:52 -0000 1.38 +++ runner.py 25 Oct 2005 01:26:10 -0000 1.39 @@ -513,6 +513,7 @@ ("master", "m", None, "Location of the buildmaster's PBListener (host:port)"), ("username", "u", None, "Username performing the commit"), + ("branch", "b", None, "Branch specifier"), ("revision", "r", None, "Revision specifier (string)"), ("revision_number", "n", None, "Revision specifier (integer)"), ("revision_file", None, None, "Filename containing revision spec"), @@ -534,6 +535,7 @@ opts = loadOptions() user = config.get('username', opts.get('username')) master = config.get('master', opts.get('master')) + branch = config.get('branch', opts.get('branch')) revision = config.get('revision') # SVN and P4 use numeric revisions if config.get("revision_number"): @@ -557,7 +559,7 @@ assert master, "you must provide the master location" s = Sender(master, user) - d = s.send(revision, comments, files) + d = s.send(branch, revision, comments, files) if runReactor: d.addCallbacks(s.printSuccess, s.printFailure) d.addCallback(s.stop) From warner at users.sourceforge.net Tue Oct 25 01:26:13 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:26:13 +0000 Subject: [Buildbot-commits] buildbot/buildbot/test test_changes.py,1.6,1.7 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8323/buildbot/test Modified Files: test_changes.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-382 Creator: Brian Warner update manual, add --branch to 'buildbot sendchange' * docs/buildbot.texinfo: update lots of stuff * buildbot/scripts/runner.py (sendchange): add a --branch argument to the 'buildbot sendchange' command * buildbot/clients/sendchange.py (Sender.send): same * buildbot/changes/pb.py (ChangePerspective): same * buildbot/test/test_changes.py (Sender.testSender): test it Index: test_changes.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_changes.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- test_changes.py 14 Oct 2005 19:47:58 -0000 1.6 +++ test_changes.py 25 Oct 2005 01:26:10 -0000 1.7 @@ -175,3 +175,18 @@ self.failUnlessEqual(c.comments, "") self.failUnlessEqual(c.revision, 42) + # verify --branch too + self.options['branch'] = "branches/test" + + d = runner.sendchange(self.options) + d.addCallback(self._testSender_6) + return d + + def _testSender_6(self, res): + self.failUnlessEqual(len(self.cm.changes), 1) + c = self.cm.changes.pop() + self.failUnlessEqual(c.who, "alice") + self.failUnlessEqual(c.files, ["foo.c"]) + self.failUnlessEqual(c.comments, "") + self.failUnlessEqual(c.revision, 42) + self.failUnlessEqual(c.branch, "branches/test") From warner at users.sourceforge.net Tue Oct 25 01:26:13 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:26:13 +0000 Subject: [Buildbot-commits] buildbot/buildbot/clients sendchange.py,1.2,1.3 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/clients In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8323/buildbot/clients Modified Files: sendchange.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-382 Creator: Brian Warner update manual, add --branch to 'buildbot sendchange' * docs/buildbot.texinfo: update lots of stuff * buildbot/scripts/runner.py (sendchange): add a --branch argument to the 'buildbot sendchange' command * buildbot/clients/sendchange.py (Sender.send): same * buildbot/changes/pb.py (ChangePerspective): same * buildbot/test/test_changes.py (Sender.testSender): test it Index: sendchange.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/clients/sendchange.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- sendchange.py 4 May 2005 07:37:27 -0000 1.2 +++ sendchange.py 25 Oct 2005 01:26:11 -0000 1.3 @@ -10,9 +10,9 @@ self.host, self.port = master.split(":") self.port = int(self.port) - def send(self, revision, comments, files): + def send(self, branch, revision, comments, files): change = {'who': self.user, 'files': files, 'comments': comments, - 'revision': revision} + 'branch': branch, 'revision': revision} f = pb.PBClientFactory() d = f.login(credentials.UsernamePassword("change", "changepw")) From warner at users.sourceforge.net Tue Oct 25 01:26:13 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:26:13 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.24,1.25 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8323/docs Modified Files: buildbot.texinfo Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-382 Creator: Brian Warner update manual, add --branch to 'buildbot sendchange' * docs/buildbot.texinfo: update lots of stuff * buildbot/scripts/runner.py (sendchange): add a --branch argument to the 'buildbot sendchange' command * buildbot/clients/sendchange.py (Sender.send): same * buildbot/changes/pb.py (ChangePerspective): same * buildbot/test/test_changes.py (Sender.testSender): test it Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.24 retrieving revision 1.25 diff -u -d -r1.24 -r1.25 --- buildbot.texinfo 24 Oct 2005 22:42:03 -0000 1.24 +++ buildbot.texinfo 25 Oct 2005 01:26:11 -0000 1.25 @@ -281,9 +281,12 @@ a minimum. In general, having more visibility into the project is always good, -and automation makes it easier for to do the right thing. When -everyone can see the status of the project, developers are encouraged -to keep the tree in good working order. +and automation makes it easier for developers to do the right thing. +When everyone can see the status of the project, developers are +encouraged to keep the tree in good working order. Unit tests that +aren't run on a regular basis tend to suffer from bitrot just like +code does: exercising them on a regular basis helps to keep them +functioning and useful. The current version of the Buildbot is additionally targeted at distributed free-software projects, where resources and platforms are @@ -420,27 +423,29 @@ @item Python: http://www.python.org -Buildbot requires python-2.2 or later, and is primarily developed against -python-2.3. The buildmaster uses generators, a feature which is not -available in python-2.1, and both master and slave require a version of -Twisted which only works with python-2.2 or later. Certain features (like -the inclusion of build logs in status emails) require python-2.2.2 or -later. +Buildbot requires python-2.2 or later, and is primarily developed +against python-2.3. The buildmaster uses generators, a feature which +is not available in python-2.1, and both master and slave require a +version of Twisted which only works with python-2.2 or later. Certain +features (like the inclusion of build logs in status emails) require +python-2.2.2 or later. The IRC ``force build'' command requires +python-2.3 (for the shlex.split function). @item Twisted: http://twistedmatrix.com Both the buildmaster and the buildslaves require Twisted-1.3.0 or -later. They have been briefly tested against Twisted-1.2.0, and might -even work with Twisted-1.1.0, but 1.3.0 is the version that has -received the most testing. +later. It has been mainly developed against Twisted-2.0.1, but has +been tested against Twisted-2.1.0 (the most recent as of this +writing), and might even work on versions as old as Twisted-1.1.0, but +as always the most recent version is recommended. -They run against Twisted-2.0.0 as well, albeit with a number of -warnings about the use of deprecated features. If you use Twisted-2.0, -you'll need at least "Twisted" (the core package), and you'll also -want TwistedMail, TwistedWeb, and TwistedWords (for sending email, -serving a web status page, and delivering build status via IRC, -respectively). +Twisted-1.3.0 and earlier were released as a single monolithic +package. When you run Buildbot against Twisted-2.0.0 or later (which +are split into a number of smaller subpackages), you'll need at least +"Twisted" (the core package), and you'll also want TwistedMail, +TwistedWeb, and TwistedWords (for sending email, serving a web status +page, and delivering build status via IRC, respectively). @end itemize Certain other packages may be useful on the system running the @@ -458,6 +463,10 @@ @end itemize +And of course, your project's build process will impose additional +requirements on the buildslaves. These hosts must have all the tools +necessary to compile and test your project's source code. + @node Installing the code, Creating a buildmaster, Requirements, Installation @section Installing the code @@ -498,21 +507,12 @@ PYTHONPATH=. trial -v buildbot.test @end example -This should run up to 109 tests, depending upon what VC tools you have -installed. On my desktop machine it takes about two minutes to complete. -Nothing should fail, a few might be skipped. If any of the tests fail, you -should stop and investigate the cause before continuing the installation -process, as it will probably be easier to track down the bug early. - -If you want to test the VC checkout process, you'll need to install a -tarball of repositories, available from http://buildbot.sf.net/ . Otherwise -there are about 8 tests which will be skipped (all with names like testSVN -and testArchHTTP). If you unpack this tarball in ~/tmp, it will create -~/tmp/buildbot-test-vc-1, and you can enable the extra tests with: - - at example -PYTHONPATH=. BUILDBOT_TEST_VC=~/tmp trial -v buildbot.test - at end example +This should run up to 175 tests, depending upon what VC tools you have +installed. On my desktop machine it takes about four minutes to +complete. Nothing should fail, a few might be skipped. If any of the +tests fail, you should stop and investigate the cause before +continuing the installation process, as it will probably be easier to +track down the bug early. If you cannot or do not wish to install the buildbot into a site-wide location like @file{/usr} or @file{/usr/local}, you can also install @@ -567,14 +567,13 @@ placed in the working directory, named @file{master.cfg.sample}, which can be copied to @file{master.cfg} and edited to suit your purposes. -Internal details: - -This command creates a file named @file{buildbot.tac} that contains -all the state necessary to create the buildmaster. Twisted has a tool -called @code{twistd} which can use this .tac file to create and launch -a buildmaster instance. twistd takes care of logging and daemonization -(running the program in the background). @file{/usr/bin/buildbot} is a -front end which runs twistd for you. +(Internal details: This command creates a file named + at file{buildbot.tac} that contains all the state necessary to create +the buildmaster. Twisted has a tool called @code{twistd} which can use +this .tac file to create and launch a buildmaster instance. twistd +takes care of logging and daemonization (running the program in the +background). @file{/usr/bin/buildbot} is a front end which runs twistd +for you.) In addition to @file{buildbot.tac}, a small @file{Makefile.sample} is installed. This can be used as the basis for customized daemon startup, @@ -669,7 +668,7 @@ @item the password assigned to your buildslave @item -the hostname and port number of the buildmaster, i.e. example.com:8007 +the hostname and port number of the buildmaster, i.e. buildbot.example.org:8007 @end itemize @item @@ -706,9 +705,9 @@ libraries installed, and finally the version of the buildbot code which is running the buildslave. -If you run many buildslaves, you may want to create - at file{~buildslave/info} and share it among all the buildslaves with -symlinks. +If you run many buildslaves, you may want to create a single + at file{~buildslave/info} file and share it among all the buildslaves +with symlinks. @end enumerate @@ -744,11 +743,6 @@ @@reboot buildbot @var{BASEDIR} @end example -There is also experimental support for sysvinit-style - at file{/etc/init.d/buildbot} startup scripts. - at file{debian/buildbot.init} and @file{debian/buildbot.default} may be -useful to look at. - It is important to remember that the environment provided to cron jobs and init scripts can be quite different that your normal runtime. There may be fewer environment variables specified, and the PATH may @@ -928,14 +922,14 @@ These source trees come from a Version Control System of some kind. CVS and Subversion are two popular ones, but the Buildbot supports -others. All VC systems at footnote{except Darcs, but since the Buildbot -never changes its local source tree we can ignore the fact that Darcs -uses a less centralized model} have some notion of an upstream - at code{repository} which acts as a server, from which clients can -obtain source trees according to various parameters. The VC repository -provides source trees of various projects, for different branches, and -from various points in time. The first thing we have to do is to -specify which source tree we want to get. +others. All VC systems have some notion of an upstream + at code{repository} which acts as a server at footnote{except Darcs, but +since the Buildbot never modifies its local source tree we can ignore +the fact that Darcs uses a less centralized model}, from which clients +can obtain source trees according to various parameters. The VC +repository provides source trees of various projects, for different +branches, and from various points in time. The first thing we have to +do is to specify which source tree we want to get. @menu * Generalizing VC Systems:: @@ -965,7 +959,7 @@ out of various source directories are more complex than we're prepared to take advantage of here}. The SHA1 revision ID used by Monotone is also a kind of revision stamp, in that it specifies a unique copy of -the source tree. +the source tree, as does a Darcs ``context'' file. When we aren't intending to make any changes to the sources we check out (at least not any that need to be committed back upstream), there are two @@ -1108,13 +1102,13 @@ @heading Who -Each Change has a @code{who} attribute, which specifies which developer -is responsible for the change. This is a string which comes from a -namespace controlled by the VC repository. Frequently this means it is -a username on the host which runs the repository, but not all VC -systems require this (Arch, for example, would use a fully-qualified - at code{Arch ID}, which looks like an email address). Each -StatusNotifier will map the @code{who} attribute into something +Each Change has a @code{who} attribute, which specifies which +developer is responsible for the change. This is a string which comes +from a namespace controlled by the VC repository. Frequently this +means it is a username on the host which runs the repository, but not +all VC systems require this (Arch, for example, uses a fully-qualified + at code{Arch ID}, which looks like an email address, as does Darcs). +Each StatusNotifier will map the @code{who} attribute into something appropriate for their particular means of communication: an email address, an IRC handle, etc. @@ -1153,9 +1147,9 @@ @item SVN @code{revision} is an int, a transation number (r%d) @item Darcs -does not yet use @code{revision} + at code{revision} is a large string, the output of @code{darcs changes --context} @item Arch/Bazaar - at code{revision} is a string, ending in --patch-%d + at code{revision} is the full revision ID (ending in --patch-%d) @item P4 @code{revision} is an int, the transaction number @end table @@ -1172,8 +1166,8 @@ subdirectories of the repository, so the file's ``svnurl'' is a combination of some base URL, the branch name, and the filename within the branch. (In a sense, the branch name and the filename inhabit the -same namespace). For Darcs, the branch name @emph{replaces} the -repository. +same namespace). Darcs branches are subdirectories of a base URL just +like SVN. @table @samp @item CVS @@ -1181,7 +1175,7 @@ @item SVN branch='branches/trunk', files=['src/foo.c'] @item Darcs -branch='http://darcs.example.org/repos/trunk', files=['src/foo.c'] +branch='branches/trunk', files=['src/foo.c'] @item Arch/Bazaar branch='buildbot--usebranches--0', files=['buildbot/master.py'] @end table @@ -1224,8 +1218,8 @@ The @code{tree-stable-timer} and @code{isFileImportant} decisions are made by the Scheduler. Dependencies are also implemented here. -Periodic builds (those which are run every N seconds rather than -because of new Changes) are triggered by a special @code{Periodic} +Periodic builds (those which are run every N seconds rather than after +new Changes arrive) are triggered by a special @code{Periodic} Scheduler subclass. The default Scheduler class can also be told to watch for specific branches, ignoring Changes on other branches. This may be useful if you have a trunk and a few release branches which @@ -1244,7 +1238,7 @@ @code{Scheduler} instances are activated by placing them in the @code{c['schedulers']} list in the buildmaster config file. Each -Scheduler gets a unique name. +Scheduler has a unique name. @node BuildSet, BuildRequest, Schedulers, Concepts @@ -1253,10 +1247,10 @@ @cindex BuildSet A @code{BuildSet} is the name given to a set of Builds that all -compile/test the same thing on multiple Builders. In general, all -these component Builds will perform the same sequence of Steps, using -the same source code, but on different platforms or against a -different set of libraries. +compile/test the same version of the tree on multiple Builders. In +general, all these component Builds will perform the same sequence of +Steps, using the same source code, but on different platforms or +against a different set of libraries. The @code{BuildSet} is tracked as a single unit, which fails if any of the component Builds have failed, and therefore can succeed only if @@ -1297,7 +1291,7 @@ @item (revision=REV, changes=None, patch=(LEVEL, DIFF)) This checks out the tree at the given revision REV, then applies a -patch (using @code{diff -pLEVEL Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14021 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-384 Creator: Brian Warner add 'branch' box to the debugclient's fake-commit command * buildbot/clients/debug.glade: add 'branch' box to fake-commit * buildbot/clients/debug.py (DebugWidget.do_commit): same. Don't send the branch= argument unless the user really provided one, to retain compatibility with older buildmasters that don't accept that argument. * buildbot/master.py (DebugPerspective.perspective_fakeChange): same Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.534 retrieving revision 1.535 diff -u -d -r1.534 -r1.535 --- ChangeLog 25 Oct 2005 01:26:11 -0000 1.534 +++ ChangeLog 25 Oct 2005 01:57:33 -0000 1.535 @@ -1,5 +1,13 @@ 2005-10-24 Brian Warner + * buildbot/clients/debug.glade: add 'branch' box to fake-commit + * buildbot/clients/debug.py (DebugWidget.do_commit): same. Don't + send the branch= argument unless the user really provided one, to + retain compatibility with older buildmasters that don't accept + that argument. + * buildbot/master.py (DebugPerspective.perspective_fakeChange): + same + * docs/buildbot.texinfo: update lots of stuff * buildbot/scripts/runner.py (sendchange): add a --branch argument From warner at users.sourceforge.net Tue Oct 25 01:57:35 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:57:35 +0000 Subject: [Buildbot-commits] buildbot/buildbot/clients debug.glade,1.1,1.2 debug.py,1.1,1.2 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/clients In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14021/buildbot/clients Modified Files: debug.glade debug.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-384 Creator: Brian Warner add 'branch' box to the debugclient's fake-commit command * buildbot/clients/debug.glade: add 'branch' box to fake-commit * buildbot/clients/debug.py (DebugWidget.do_commit): same. Don't send the branch= argument unless the user really provided one, to retain compatibility with older buildmasters that don't accept that argument. * buildbot/master.py (DebugPerspective.perspective_fakeChange): same Index: debug.glade =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/clients/debug.glade,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- debug.glade 22 Apr 2005 02:57:23 -0000 1.1 +++ debug.glade 25 Oct 2005 01:57:33 -0000 1.2 @@ -17,6 +17,7 @@ False GDK_WINDOW_TYPE_HINT_NORMAL GDK_GRAVITY_NORTH_WEST + True @@ -60,6 +61,10 @@ 0.5 0 0 + PANGO_ELLIPSIZE_NONE + -1 + False + 0 0 @@ -198,7 +203,7 @@ 0 twisted/internet/app.py True - * + * False @@ -234,6 +239,10 @@ 0.5 0 0 + PANGO_ELLIPSIZE_NONE + -1 + False + 0 0 @@ -251,7 +260,59 @@ 0 bob True - * + * + False + + + 0 + True + True + + + + + 0 + True + True + + + + + + True + False + 0 + + + + True + True + Branch: + True + GTK_RELIEF_NORMAL + True + False + False + True + + + + 0 + False + False + + + + + + True + True + True + True + 0 + + True + * False @@ -303,7 +364,7 @@ 0 True - * + * False @@ -337,6 +398,10 @@ 0.5 2 0 + PANGO_ELLIPSIZE_NONE + -1 + False + 0 label_item @@ -383,6 +448,10 @@ 0.5 0 0 + PANGO_ELLIPSIZE_NONE + -1 + False + 0 0 @@ -400,7 +469,7 @@ 0 one True - * + * False @@ -475,6 +544,10 @@ 0.5 7 0 + PANGO_ELLIPSIZE_NONE + -1 + False + 0 0 @@ -573,6 +646,10 @@ 0.5 2 0 + PANGO_ELLIPSIZE_NONE + -1 + False + 0 label_item Index: debug.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/clients/debug.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- debug.py 22 Apr 2005 02:57:23 -0000 1.1 +++ debug.py 25 Oct 2005 01:57:33 -0000 1.2 @@ -36,6 +36,8 @@ c('do_poke_irc', self.do_poke_irc) c('do_build', self.do_build) c('do_commit', self.do_commit) + c('on_usebranch_toggled', self.usebranch_toggled) + self.usebranch_toggled(g('usebranch')) c('on_userevision_toggled', self.userevision_toggled) self.userevision_toggled(g('userevision')) c('do_current_offline', self.do_current, "offline") @@ -92,6 +94,13 @@ d = self.remote.callRemote("forceBuild", name) d.addErrback(self.err) + def usebranch_toggled(self, widget): + rev = self.xml.get_widget('branch') + if widget.get_active(): + rev.set_sensitive(True) + else: + rev.set_sensitive(False) + def userevision_toggled(self, widget): rev = self.xml.get_widget('revision') if widget.get_active(): @@ -104,6 +113,13 @@ return filename = self.filename.get_text() who = self.xml.get_widget("who").get_text() + + branch = None + if self.xml.get_widget("usebranch").get_active(): + branch = self.xml.get_widget('branch').get_text() + if branch == '': + branch = None + revision = None if self.xml.get_widget("userevision").get_active(): revision = self.xml.get_widget('revision').get_text() @@ -111,8 +127,13 @@ revision = int(revision) except ValueError: pass - d = self.remote.callRemote("fakeChange", filename, - revision=revision, who=who) + if revision == '': + revision = None + + kwargs = { 'revision': revision, 'who': who } + if branch: + kwargs['branch'] = branch + d = self.remote.callRemote("fakeChange", filename, **kwargs) d.addErrback(self.err) def do_current(self, widget, state): From warner at users.sourceforge.net Tue Oct 25 01:57:35 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 01:57:35 +0000 Subject: [Buildbot-commits] buildbot/buildbot master.py,1.83,1.84 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14021/buildbot Modified Files: master.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-384 Creator: Brian Warner add 'branch' box to the debugclient's fake-commit command * buildbot/clients/debug.glade: add 'branch' box to fake-commit * buildbot/clients/debug.py (DebugWidget.do_commit): same. Don't send the branch= argument unless the user really provided one, to retain compatibility with older buildmasters that don't accept that argument. * buildbot/master.py (DebugPerspective.perspective_fakeChange): same Index: master.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/master.py,v retrieving revision 1.83 retrieving revision 1.84 diff -u -d -r1.83 -r1.84 --- master.py 23 Oct 2005 05:06:02 -0000 1.83 +++ master.py 25 Oct 2005 01:57:33 -0000 1.84 @@ -445,9 +445,10 @@ bc = c.getBuilder(buildername) bc.forceBuild(who, "debug tool 'Force Build' button pushed") - def perspective_fakeChange(self, file, revision=None, who="fakeUser"): + def perspective_fakeChange(self, file, revision=None, who="fakeUser", + branch=None): change = Change(who, [file], "some fake comments\n", - revision=revision) + branch=branch, revision=revision) c = interfaces.IControl(self.master) c.addChange(change) From warner at users.sourceforge.net Tue Oct 25 02:12:12 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 02:12:12 +0000 Subject: [Buildbot-commits] buildbot/buildbot/scripts tryclient.py,1.9,1.10 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot/scripts In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16562/buildbot/scripts Modified Files: tryclient.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-386 Creator: Brian Warner minor comments Index: tryclient.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/tryclient.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- tryclient.py 31 Aug 2005 01:51:25 -0000 1.9 +++ tryclient.py 25 Oct 2005 02:12:10 -0000 1.10 @@ -350,7 +350,11 @@ # may emit status messages while we wait wait = bool(self.getopt("wait", "try_wait", False)) if not wait: - # TODO: emit the URL where they can follow the builds + # TODO: emit the URL where they can follow the builds. This + # requires contacting the Status server over PB and doing + # getURLForThing() on the BuildSetStatus. To get URLs for + # individual builds would require we wait for the builds to + # start. print "not waiting for builds to finish" return d = self.running = defer.Deferred() From warner at users.sourceforge.net Tue Oct 25 02:14:25 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 02:14:25 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.25,1.26 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16980/docs Modified Files: buildbot.texinfo Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-388 Creator: Brian Warner 'buildbot sendchange' has --wait=False as the default Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- buildbot.texinfo 25 Oct 2005 01:26:11 -0000 1.25 +++ buildbot.texinfo 25 Oct 2005 02:14:23 -0000 1.26 @@ -3696,7 +3696,7 @@ updated), and a patch that can be applied to that revision of the tree to make it match the developer's copy. This (revision, patch) pair is then sent to the buildmaster, which runs a build with that -SourceStamp. By default, the tool will emit status messages as the +SourceStamp. If you want, the tool will emit status messages as the builds run, and will not terminate until the first failure has been detected (or the last success). From warner at users.sourceforge.net Tue Oct 25 02:49:56 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 02:49:56 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.26,1.27 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21959/docs Modified Files: buildbot.texinfo Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-390 Creator: Brian Warner prepare for release * README: update for 0.7.0 * NEWS: same * docs/buildbot.texinfo: move the freshcvs stuff out of the README Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- buildbot.texinfo 25 Oct 2005 02:14:23 -0000 1.26 +++ buildbot.texinfo 25 Oct 2005 02:49:54 -0000 1.27 @@ -2087,6 +2087,38 @@ @end table + at heading Example + +To set up the freshCVS server, add a statement like the following to +your @file{freshCfg} file: + + at example +pb = ConfigurationSet([ + (None, None, None, PBService(userpass=('foo', 'bar'), port=4519)), + ]) + at end example + +This will announce all changes to a client which connects to port 4519 +using a username of 'foo' and a password of 'bar'. + +Then add a clause like this to your buildmaster's @file{master.cfg}: + + at example +BuildmasterConfig['sources'] = [FreshCVSSource("cvs.example.com", 4519, + "foo", "bar", + prefix="glib/")] + at end example + +where "cvs.example.com" is the host that is running the FreshCVS daemon, and +"glib" is the top-level directory (relative to the repository's root) where +all your source code lives. Most projects keep one or more projects in the +same repository (along with CVSROOT/ to hold admin files like loginfo and +freshCfg); the prefix= argument tells the buildmaster to ignore everything +outside that directory, and to strip that common prefix from all pathnames +it handles. + + + @node CVSToys - mail notification, Other mail notification ChangeSources, CVSToys - PBService, Change Sources @subsection CVSToys - mail notification From warner at users.sourceforge.net Tue Oct 25 02:49:56 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 02:49:56 +0000 Subject: [Buildbot-commits] buildbot README,1.27,1.28 ChangeLog,1.535,1.536 NEWS,1.46,1.47 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21959 Modified Files: README ChangeLog NEWS Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-390 Creator: Brian Warner prepare for release * README: update for 0.7.0 * NEWS: same * docs/buildbot.texinfo: move the freshcvs stuff out of the README Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.535 retrieving revision 1.536 diff -u -d -r1.535 -r1.536 --- ChangeLog 25 Oct 2005 01:57:33 -0000 1.535 +++ ChangeLog 25 Oct 2005 02:49:53 -0000 1.536 @@ -1,5 +1,9 @@ 2005-10-24 Brian Warner + * README: update for 0.7.0 + * NEWS: same + * docs/buildbot.texinfo: move the freshcvs stuff out of the README + * buildbot/clients/debug.glade: add 'branch' box to fake-commit * buildbot/clients/debug.py (DebugWidget.do_commit): same. Don't send the branch= argument unless the user really provided one, to Index: NEWS =================================================================== RCS file: /cvsroot/buildbot/buildbot/NEWS,v retrieving revision 1.46 retrieving revision 1.47 diff -u -d -r1.46 -r1.47 --- NEWS 24 Oct 2005 03:48:00 -0000 1.46 +++ NEWS 25 Oct 2005 02:49:53 -0000 1.47 @@ -1,19 +1,19 @@ User visible changes in Buildbot. -* Release x.x.x (xx) +* Release 0.7.0 (24 Oct 2005) -* new features +** new features -** new c['schedulers'] config-file element (REQUIRED) +*** new c['schedulers'] config-file element (REQUIRED) The code which decides exactly *when* a build is performed has been massively -refactored. YOU MUST UPDATE your master.cfg files to match: in general this -will merely require you to add an appropriate c['schedulers'] entry. Any old -".treeStableTime" settings on the BuildFactory instances will now be ignored. -The user's manual has complete details with examples of how the new Scheduler -classes work. +refactored, enabling much more flexible build scheduling. YOU MUST UPDATE +your master.cfg files to match: in general this will merely require you to +add an appropriate c['schedulers'] entry. Any old ".treeStableTime" settings +on the BuildFactory instances will now be ignored. The user's manual has +complete details with examples of how the new Scheduler classes work. -** c['interlocks'] removed, Locks and Dependencies now separate items +*** c['interlocks'] removed, Locks and Dependencies now separate items The c['interlocks'] config element has been removed, and its functionality replaced with two separate objects. Locks are used to tell the buildmaster @@ -23,7 +23,8 @@ otherwise your developers will be suffering from the same limitations). The Lock object is created in the config file and then referenced by a Step specification tuple or by the 'locks' key of the Builder specification -dictionary. +dictionary. Locks come in two flavors: MasterLocks are buildmaster-wide, +while SlaveLocks are specific to a single buildslave. When you want to have one Build run or not run depending upon whether some other set of Builds have passed or failed, you use a special kind of @@ -34,13 +35,13 @@ Both features are fully documented in the user's manual. -** 'try' +*** 'buildbot try' -The 'buildbot try' feature has finally been added. There is some -configuration involved, both in the buildmaster config and on the developer's -side, but once in place this allows the developer to type 'buildbot try' in -their locally-modified tree and to be given a report of what would happen if -their changes were to be committed. This works by computing a (base revision, +The 'try' feature has finally been added. There is some configuration +involved, both in the buildmaster config and on the developer's side, but +once in place this allows the developer to type 'buildbot try' in their +locally-modified tree and to be given a report of what would happen if their +changes were to be committed. This works by computing a (base revision, patch) tuple that describes the developer's tree, sending that to the buildmaster, then running a build with that source on a given set of Builders. The 'buildbot try' tool then emits status messages until the builds @@ -54,28 +55,32 @@ Instructions for developers who want to use 'try' (and the configuration changes necessary to enable its use) are in the user's manual. -** Build-On-Branch +*** Build-On-Branch When suitably configured, the buildbot can be used to build trees from a variety of related branches. You can set up Schedulers to build a tree using whichever branch was last changed, or users can request builds of specific -branches through IRC, the web page, or the CLI 'buildbot force' subcommand. +branches through IRC, the web page, or (eventually) the CLI 'buildbot force' +subcommand. The IRC 'force' command now takes --branch and --revision arguments (not that they always make sense). Likewise the HTML 'force build' button now has an input field for branch and revision. Your build's source-checkout step must be suitably configured to support this: for SVN it involves giving both a base URL and a default branch. Other VC systems are configured differently. +The ChangeSource must also provide branch information: the 'buildbot +sendchange' command now takes a --branch argument to help hook script writers +accomplish this. -** Multiple slaves per Builder +*** Multiple slaves per Builder You can now attach multiple buildslaves to each Builder. This can provide -redundancy or load-balancing among many machines equally capable of running -the build. To use this, define a key in the Builder specification dictionary -named 'slavenames' with a list of buildslave names (instead of the usual -'slavename' that contains just a single slavename). +redundancy or primitive load-balancing among many machines equally capable of +running the build. To use this, define a key in the Builder specification +dictionary named 'slavenames' with a list of buildslave names (instead of the +usual 'slavename' that contains just a single slavename). -** minor new features +*** minor new features The IRC and email status-reporting facilities now provide more specific URLs for particular builds, in addition to the generic buildmaster home page. The @@ -99,7 +104,7 @@ ** creeping version dependencies -The IRC 'force build' command requires python2.3 (for the shlex.split +The IRC 'force build' command now requires python2.3 (for the shlex.split function). Index: README =================================================================== RCS file: /cvsroot/buildbot/buildbot/README,v retrieving revision 1.27 retrieving revision 1.28 diff -u -d -r1.27 -r1.28 --- README 18 May 2005 08:01:36 -0000 1.27 +++ README 25 Oct 2005 02:49:53 -0000 1.28 @@ -56,19 +56,20 @@ available in python-2.1, and both master and slave require a version of Twisted which only works with python-2.2 or later. Certain features (like the inclusion of build logs in status emails) require python-2.2.2 or - later. + later, while the IRC 'force' command requires python-2.3 . Twisted: http://twistedmatrix.com Both the buildmaster and the buildslaves require Twisted-1.3.0 or later. - They have been briefly tested against Twisted-1.2.0, and might even work - with Twisted-1.1.0, but 1.3.0 is the version that has received the most - testing. + It has been mainly developed against Twisted-2.0.1, but has been tested + against Twisted-2.1.0 (the most recent as this time), and might even work + on versions as old as Twisted-1.1.0, but as always the most recent version + is recommended. - Both work with Twisted-2.0.0 as well. You'll need at least "Twisted" (the - core package), and you'll also want TwistedMail, TwistedWeb, and - TwistedWords (for sending email, serving a web status page, and delivering - build status via IRC, respectively). + When using the split subpackages of Twisted-2.x.x, you'll need at least + "Twisted" (the core package), and you'll also want TwistedMail, + TwistedWeb, and TwistedWords (for sending email, serving a web status + page, and delivering build status via IRC, respectively). CVSToys: http://purl.net/net/CVSToys @@ -80,8 +81,9 @@ INSTALLATION: -Please read the User's Manual in docs/buildbot.info for complete -instructions. +Please read the User's Manual in docs/buildbot.info (or in HTML form on the +buildbot web site) for complete instructions. This file only contains a brief +summary. RUNNING THE UNIT TESTS @@ -89,20 +91,12 @@ PYTHONPATH=. trial -v buildbot.test -This should run up to 124 tests, depending upon what VC tools you have -installed. On my desktop machine it takes about two minutes to complete. +This should run up to 175 tests, depending upon what VC tools you have +installed. On my desktop machine it takes about four minutes to complete. Nothing should fail, a few might be skipped. If any of the tests fail, you should stop and investigate the cause before continuing the installation process, as it will probably be easier to track down the bug early. -If you want to test the VC checkout process, you'll need to install a -tarball of repositories, available from http://buildbot.sf.net/ . Otherwise -there are about 8 tests which will be skipped (all with names like testSVN -and testArchHTTP). If you unpack this tarball in ~/tmp, it will create -~/tmp/buildbot-test-vc-1, and you can enable the extra tests with: - - PYTHONPATH=. BUILDBOT_TEST_VC=~/tmp trial -v buildbot.test - INSTALLING THE LIBRARIES: @@ -138,13 +132,14 @@ buildbot slave WORKDIR MASTERHOST:PORT SLAVENAME PASSWORD -This will create a "TAP" file called "buildbot.tap", which bundles up all -the state needed by the build slave application. Twisted has a tool called -"twistd" which knows how to load these saved applications and start running -them. twistd takes care of logging and daemonization (running the program in -the background). /usr/bin/buildbot is a front end which runs twistd for you. +This will create a file called "buildbot.tac", which bundles up all the state +needed by the build slave application. Twisted has a tool called "twistd" +which knows how to load these saved applications and start running them. +twistd takes care of logging and daemonization (running the program in the +background). /usr/bin/buildbot is a front end which runs twistd for you. -Once you have the .tap file, you start it running like this: +Once you've set up the directory with the .tac file, you start it running +like this: buildbot start WORKDIR @@ -180,65 +175,14 @@ SETTING UP A BUILD MASTER: -First, read through the rest of the documents to understand the definition -of Builds and BuildFactories. Look at the docs in docs/*.xhtml. - -Second, know that this should get easier in the future, probably with a -web-based interface to create new kinds of Builds and manipulate them. - -You start by picking a base directory where the buildmaster will store all -its status and logfiles. Then you create the buildmaster's buildbot.tap file -with the 'buildbot' tool: - - buildbot master WORKDIR - -This will also create a sample configuration file for you in -WORKDIR/master.cfg . Edit this to describe how your buildmaster should -operate: what port it should listen on for connections from the build slaves, -which build slaves are allowed to connect, how to watch for changes to the -source tree, how to run builds, and how to deliver status information. - -There are more examples in docs/examples/, and plenty of documentation on the -various settings in docs/ . Everything is controlled by the config file. It -must be readable from the master's basedir under the filename 'master.cfg'. - -Then you launch the buildmaster just as you would launch the buildslaves: the -'buildbot' tool runs twistd, which automatically forks off the daemon into -the background: - - buildbot start WORKDIR - -All buildmaster output is logged into 'twistd.log' in that same directory. - -The buildmaster will read master.cfg, set up the builders, start listening -on the web server port and begin accepting connections from the buildslaves. - - - CONNECTING TO A FRESHCVS DAEMON: - -Set up CVSToys-1.0.10, and add a statement like the following to your -freshCfg file: - -pb = ConfigurationSet([ - (None, None, None, PBService(userpass=('foo', 'bar'), port=4519)), - ]) - -This will announce all changes to a client which connects to port 4519 using -a username of 'foo' and a password of 'bar'. - -Then add a clause like this to your buildmaster's master.cfg: - -BuildmasterConfig['sources'] = [FreshCVSSource("cvs.example.com", 4519, - "foo", "bar", - prefix="glib/")] +Please read the user's manual for instructions. The short form is that you +use 'buildbot master MASTERDIR' to create the base directory, then you edit +the 'master.cfg' file to configure the buildmaster. Once this is ready, you +use 'buildbot START MASTERDIR' to launch it. -where "cvs.example.com" is the host that is running the FreshCVS daemon, and -"glib" is the top-level directory (relative to the repository's root) where -all your source code lives. Most projects keep one or more projects in the -same repository (along with CVSROOT/ to hold admin files like loginfo and -freshCfg); the prefix= argument tells the buildmaster to ignore everything -outside that directory, and to strip that common prefix from all pathnames -it handles. +A sample configuration file will be created for you in WORKDIR/master.cfg . +There are more examples in docs/examples/, and plenty of documentation in the +user's manual. Everything is controlled by the config file. SUPPORT: From warner at users.sourceforge.net Tue Oct 25 02:56:35 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 02:56:35 +0000 Subject: [Buildbot-commits] buildbot/buildbot __init__.py,1.16,1.17 Message-ID: Update of /cvsroot/buildbot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23055/buildbot Modified Files: __init__.py Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-392 Creator: Brian Warner releasing buildbot-0.7.0 * buildbot/__init__.py (version): Releasing buildbot-0.7.0 * docs/buildbot.texinfo: set version number to match Index: __init__.py =================================================================== RCS file: /cvsroot/buildbot/buildbot/buildbot/__init__.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- __init__.py 24 May 2005 18:57:49 -0000 1.16 +++ __init__.py 25 Oct 2005 02:56:32 -0000 1.17 @@ -1,3 +1,3 @@ #! /usr/bin/python -version = "0.6.6+" +version = "0.7.0" From warner at users.sourceforge.net Tue Oct 25 02:56:35 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 02:56:35 +0000 Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.27,1.28 Message-ID: Update of /cvsroot/buildbot/buildbot/docs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23055/docs Modified Files: buildbot.texinfo Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-392 Creator: Brian Warner releasing buildbot-0.7.0 * buildbot/__init__.py (version): Releasing buildbot-0.7.0 * docs/buildbot.texinfo: set version number to match Index: buildbot.texinfo =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v retrieving revision 1.27 retrieving revision 1.28 diff -u -d -r1.27 -r1.28 --- buildbot.texinfo 25 Oct 2005 02:49:54 -0000 1.27 +++ buildbot.texinfo 25 Oct 2005 02:56:33 -0000 1.28 @@ -1,7 +1,7 @@ \input texinfo @c -*-texinfo-*- @c %**start of header @setfilename buildbot.info - at settitle BuildBot Manual x.x + at settitle BuildBot Manual 0.7.0 @c %**end of header @copying From warner at users.sourceforge.net Tue Oct 25 02:56:35 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 02:56:35 +0000 Subject: [Buildbot-commits] buildbot ChangeLog,1.536,1.537 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23055 Modified Files: ChangeLog Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-392 Creator: Brian Warner releasing buildbot-0.7.0 * buildbot/__init__.py (version): Releasing buildbot-0.7.0 * docs/buildbot.texinfo: set version number to match Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v retrieving revision 1.536 retrieving revision 1.537 diff -u -d -r1.536 -r1.537 --- ChangeLog 25 Oct 2005 02:49:53 -0000 1.536 +++ ChangeLog 25 Oct 2005 02:56:33 -0000 1.537 @@ -1,5 +1,10 @@ 2005-10-24 Brian Warner + * buildbot/__init__.py (version): Releasing buildbot-0.7.0 + * docs/buildbot.texinfo: set version number to match + +2005-10-24 Brian Warner + * README: update for 0.7.0 * NEWS: same * docs/buildbot.texinfo: move the freshcvs stuff out of the README From warner at users.sourceforge.net Tue Oct 25 03:15:36 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 03:15:36 +0000 Subject: [Buildbot-commits] buildbot Makefile,1.15,1.16 Message-ID: Update of /cvsroot/buildbot/buildbot In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28074 Modified Files: Makefile Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-394 Creator: Brian Warner fix some release-related targets in the Makefiles --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: Makefile =================================================================== RCS file: /cvsroot/buildbot/buildbot/Makefile,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- Makefile 16 Aug 2005 00:23:53 -0000 1.15 +++ Makefile 25 Oct 2005 03:15:34 -0000 1.16 @@ -44,4 +44,5 @@ chmod a+x bin/buildbot contrib/*.py rm -rf _trial_temp python ./setup.py clean + rm -f MANIFEST python ./setup.py sdist From warner at users.sourceforge.net Tue Oct 25 03:15:36 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 03:15:36 +0000 Subject: [Buildbot-commits] buildbot/docs/PyCon-2003 Makefile,1.2,1.3 Message-ID: Update of /cvsroot/buildbot/buildbot/docs/PyCon-2003 In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28074/docs/PyCon-2003 Modified Files: Makefile Log Message: Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-394 Creator: Brian Warner fix some release-related targets in the Makefiles --This line, and those below, will be ignored-- Files to commit: This list might be incomplete or outdated if editing the log message was not invoked from an up-to-date changes buffer! Index: Makefile =================================================================== RCS file: /cvsroot/buildbot/buildbot/docs/PyCon-2003/Makefile,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- Makefile 1 Oct 2004 02:36:35 -0000 1.2 +++ Makefile 25 Oct 2005 03:15:34 -0000 1.3 @@ -32,4 +32,4 @@ done %.png: source/%.svg - sodipodi --export-png $@ $< + inkscape --export-png $@ $< From warner at users.sourceforge.net Tue Oct 25 03:53:22 2005 From: warner at users.sourceforge.net (Brian Warner) Date: Tue, 25 Oct 2005 03:53:22 +0000 Subject: [Buildbot-commits] site manual-0.7.0.html,NONE,1.1 ChangeLog,1.22,1.23 NEWS,1.8,1.9 README,1.14,1.15 index.html,1.48,1.49 manual-CVS.html,1.3,1.4 Message-ID: Update of /cvsroot/buildbot/site In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv937 Modified Files: ChangeLog NEWS README index.html manual-CVS.html Added Files: manual-0.7.0.html Log Message: update for 0.7.0 Index: ChangeLog =================================================================== RCS file: /cvsroot/buildbot/site/ChangeLog,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- ChangeLog 23 May 2005 23:53:55 -0000 1.22 +++ ChangeLog 25 Oct 2005 03:53:20 -0000 1.23 @@ -1,3 +1,901 @@ +2005-10-24 Brian Warner + + * buildbot/__init__.py (version): Releasing buildbot-0.7.0 + * docs/buildbot.texinfo: set version number to match + +2005-10-24 Brian Warner + + * README: update for 0.7.0 + * NEWS: same + * docs/buildbot.texinfo: move the freshcvs stuff out of the README + + * buildbot/clients/debug.glade: add 'branch' box to fake-commit + * buildbot/clients/debug.py (DebugWidget.do_commit): same. Don't + send the branch= argument unless the user really provided one, to + retain compatibility with older buildmasters that don't accept + that argument. + * buildbot/master.py (DebugPerspective.perspective_fakeChange): + same + + * docs/buildbot.texinfo: update lots of stuff + + * buildbot/scripts/runner.py (sendchange): add a --branch argument + to the 'buildbot sendchange' command + * buildbot/clients/sendchange.py (Sender.send): same + * buildbot/changes/pb.py (ChangePerspective): same + * buildbot/test/test_changes.py (Sender.testSender): test it + + * buildbot/process/step.py (SVN.__init__): change 'base_url' and + 'default_branch' argument names to 'baseURL' and 'defaultBranch', + for consistency with other BuildStep arguments that use camelCase. + Well, at least more of them use camelCase (like flunkOnWarnings) + than don't.. I wish I'd picked one style and stuck with it + earlier. Annoying, but it's best done before the release, since + these arguments didn't exist at all in 0.6.6 . + (Darcs): same + * buildbot/test/test_vc.py (SVN.testCheckout): same + (Darcs.testPatch): same + * docs/buildbot.texinfo (SVN): document the change + (Darcs): same, add some build-on-branch docs + * docs/examples/twisted_master.cfg: match change + + * buildbot/process/step.py (BuildStep): rename + slaveVersionNewEnough to slaveVersionIsOlderThan, because that's + how it is normally used. + * buildbot/test/test_steps.py (Version.checkCompare): same + + * buildbot/process/step.py (CVS.startVC): refuse to build + update/copy -style builds on a non-default branch with an old + buildslave (<=0.6.6) that doesn't know how to do it properly. The + concern is that it will do a VC 'update' in an existing tree when + it is supposed to be switching branches (and therefore clobbering + the tree to do a full checkout), thus building the wrong source. + This used to be a warning, but I think the confusion it is likely + to cause warrants making it an error. + (SVN.startVC): same, also make mode=export on old slaves an error + (Darcs.startVC): same + (Git.startVC): improve error message for non-Git-enabled slaves + (Arch.checkSlaveVersion): same. continue to emit a warning when a + specific revision is built on a slave that doesn't pay attention + to args['revision'], because for slowly-changing trees it will + probably do the right thing, and because we have no way to tell + whether we're asking it to build the most recent version or not. + * buildbot/interfaces.py (BuildSlaveTooOldError): new exception + + * buildbot/scripts/runner.py (SlaveOptions.postOptions): assert + that 'master' is in host:portnum format, to catch errors sooner + +2005-10-23 Brian Warner + + * buildbot/process/step_twisted.py (ProcessDocs.createSummary): + when creating the list of warning messages, include the line + immediately after each WARNING: line, since that's usually where + the file and line number wind up. + + * docs/examples/twisted_master.cfg: OS-X slave now does QT, add a + TryScheduler + + * NEWS: update + +2005-10-22 Brian Warner + + * buildbot/status/html.py (HtmlResource): incorporate valid-HTML + patch from Brad Hards + * buildbot/status/classic.css: same + * buildbot/test/test_web.py (Waterfall): match changes + + * buildbot/test/test_steps.py (BuildStep.setUp): set + nextBuildNumber so the test passes + * buildbot/test/test_status.py (MyBuilder): same + + * buildbot/status/html.py (StatusResourceBuild.body): revision + might be numeric, so stringify it before html-escapifying it + (CurrentBox.getBox): add a "waiting" state, and show a countdown + timer for the upcoming build + * buildbot/status/classic.css: add background-color attributes for + offline/waiting/building classes + + * buildbot/status/builder.py (BuildStatus): derive from + styles.Versioned, fix upgrade of .sourceStamp attribute. Also set + the default (i.e. unknown) .slavename to "???" instead of None, + since even unknown slavenames need to be printed eventually. + (BuilderStatus): also derive from styles.Versioned . More + importantly, determine .nextBuildNumber at creation/unpickling + time by scanning the directory of saved BuildStatus instances and + choosing one larger than the highest-numbered one found. This + should fix the problem where random errors during upgrades cause + the buildbot to forget about earlier builds. .nextBuildNumber is + no longer stored in the pickle. + (Status.builderAdded): if we can't unpickle the BuilderStatus, + at least log the error. Also call Builder.determineNextBuildNumber + once the basedir is set. + + * buildbot/master.py (BuildMaster.loadChanges): do + styles.doUpgrade afterwards, in case I decide to make Changes + derived from styles.Versioned some day and forget to make this + change later. + + + * buildbot/test/test_runner.py (Options.testForceOptions): skip + when running under older pythons (<2.3) in which the shlex module + doesn't have a 'split' function. + + * buildbot/process/step.py (ShellCommand.start): make + errorMessages= be a list of strings to stuff in the log before the + command actually starts. This makes it easier to flag multiple + warning messages, e.g. when the Source steps have to deal with an + old buildslave. + (CVS.startVC): handle slaves that don't handle multiple branches + by switching into 'clobber' mode + (SVN.startVC): same. Also reject branches without base_url + (Darcs.startVC): same. Also reject revision= in older slaves + (Arch.checkSlaveVersion): same (just the multiple-branches stuff) + (Bazaar.startVC): same, and test for baz separately than for arch + + * buildbot/slave/commands.py (cvs_ver): document new features + + * buildbot/process/step.py (BuildStep.slaveVersion): document it + (BuildStep.slaveVersionNewEnough): more useful utility method + * buildbot/test/test_steps.py (Version): start testing it + + * buildbot/status/words.py (IrcStatusBot.command_FORCE): note that + the 'force' command requires python2.3, for the shlex.split method + + * docs/examples/twisted_master.cfg: remove old freshcvs stuff, + since we don't use it anymore. The Twisted buildbot uses a + PBChangeSource now. + +2005-10-21 Brian Warner + + * buildbot/process/process_twisted.py: rework all BuildFactory + classes to take a 'source' step as an argument, instead of + building up the SVN instance in the factory. + * docs/examples/twisted_master.cfg: enable build-on-branch by + providing a base_url and default_branch + + * buildbot/status/words.py (IrcStatusBot.command_FORCE): add + control over --branch and --revision, not that they are always + legal to provide + * buildbot/status/html.py (StatusResourceBuilder.force): same + (StatusResourceBuild.body): display SourceStamp components + + * buildbot/scripts/runner.py (ForceOptions): option parser for the + IRC 'force' command, so it can be shared with an eventual + command-line-tool 'buildbot force' mode. + * buildbot/test/test_runner.py (Options.testForceOptions): test it + +2005-10-20 Brian Warner + + * buildbot/status/mail.py (MailNotifier.buildMessage): reformat + + * docs/examples/twisted_master.cfg: update to use Schedulers + + * buildbot/scripts/sample.cfg: update with Schedulers + + * buildbot/interfaces.py (IBuilderControl.requestBuildSoon): new + method specifically for use by HTML "force build" button and the + IRC "force" command. Raises an immediate error if there are no + slaves available. + (IBuilderControl.requestBuild): make this just submit a build, not + try to check for existing slaves or set up any when-finished + Deferreds or anything. + * buildbot/process/builder.py (BuilderControl): same + * buildbot/status/html.py (StatusResourceBuilder.force): same + * buildbot/status/words.py (IrcStatusBot.command_FORCE): same + * buildbot/test/test_slaves.py: same + * buildbot/test/test_web.py: same + +2005-10-19 Brian Warner + + * docs/examples/twisted_master.cfg: re-sync with reality: bring + back python2.2 tests, turn off OS-X threadedselect-reactor tests + +2005-10-18 Brian Warner + + * buildbot/status/html.py: provide 'status' argument to most + StatusResourceFOO objects + (StatusResourceBuild.body): href-ify the Builder name, add "Steps + and Logfiles" section to make the Build page into a more-or-less + comprehensive source of status information about the build + + * buildbot/status/mail.py (MailNotifier): include the Build's URL + * buildbot/status/words.py (IrcStatusBot.buildFinished): same + +2005-10-17 Brian Warner + + * buildbot/process/process_twisted.py (TwistedTrial): update Trial + arguments to accomodate Twisted >=2.1.0 . I will have to figure + out what to do about other projects: the correct options for + recent Twisteds will not work for older ones. + +2005-10-15 Brian Warner + + * buildbot/status/builder.py (Status.getURLForThing): add method + to provide a URL for arbitrary IStatusFoo objects. The idea is to + use this in email/IRC status clients to make them more useful, by + providing the end user with hints on where to learn more about the + object being reported on. + * buildbot/test/test_web.py (GetURL): tests for it + +2005-10-14 Brian Warner + + * buildbot/test/test_config.py (ConfigTest._testSources_1): oops, + fix bug resulting from deferredResult changes + +2005-10-13 Brian Warner + + * buildbot/test/test_changes.py: remove use of deferredResult + * buildbot/test/test_config.py: same + * buildbot/test/test_control.py: same + * buildbot/test/test_status.py: same + * buildbot/test/test_vc.py: this is the only remaining use, since + it gets used at module level. This needs to be replaced by some + sort of class-level run-once routine. + + * buildbot/status/words.py (IrcStatusBot.command_WATCH): fix typo + + * lots: implement multiple slaves per Builder, which means multiple + current builds per Builder. Some highlights: + * buildbot/interfaces.py (IBuilderStatus.getState): return a tuple + of (state,currentBuilds) instead of (state,currentBuild) + (IBuilderStatus.getCurrentBuilds): replace getCurrentBuild() + (IBuildStatus.getSlavename): new method, so you can tell which + slave got used. This only gets set when the build completes. + (IBuildRequestStatus.getBuilds): new method + + * buildbot/process/builder.py (SlaveBuilder): add a .state + attribute to track things like ATTACHING and IDLE and BUILDING, + instead of.. + (Builder): .. the .slaves attribute here, which has been turned + into a simple list of available slaves. Added a separate + attaching_slaves list to track ones that are not yet ready for + builds. + (Builder.fireTestEvent): put off the test-event callback for a + reactor turn, to make tests a bit more consistent. + (Ping): cleaned up the slaveping a bit, now it disconnects if the + ping fails due to an exception. This needs work, I'm worried that + a code error could lead to a constantly re-connecting slave. + Especially since I'm trying to move to a distinct remote_ping + method, separate from the remote_print that we currently use. + (BuilderControl.requestBuild): return a convenience Deferred that + provides an IBuildStatus when the build finishes. + (BuilderControl.ping): ping all connected slaves, only return True + if they all respond. + + * buildbot/slave/bot.py (BuildSlave.stopService): stop trying to + reconnect when we shut down. + + * buildbot/status/builder.py: implement new methods, convert + one-build-at-a-time methods to handle multiple builds + * buildbot/status/*.py: do the same in all default status targets + * buildbot/status/html.py: report the build's slavename in the + per-Build page, report all buildslaves on the per-Builder page + + * buildbot/test/test_run.py: update/create tests + * buildbot/test/test_slaves.py: same + * buildbot/test/test_scheduler.py: remove stale test + + * docs/buildbot.texinfo: document the new builder-specification + 'slavenames' parameter + +2005-10-12 Brian Warner + + * buildbot/buildset.py (BuildSet): fix bug where BuildSet did not + report failure correctly, causing Dependent builds to run when + they shouldn't have. + * buildbot/status/builder.py (BuildSetStatus): same + * buildbot/test/test_buildreq.py (Set.testBuildSet): verify it + (Set.testSuccess): test the both-pass case too + * buildbot/test/test_dependencies.py (Dependencies.testRun_Fail): + fix this test: it was ending too early, masking the failure before + (Logger): specialized StatusReceiver to make sure the dependent + builds aren't even started, much less completed. + +2005-10-07 Brian Warner + + * buildbot/slave/bot.py (SlaveBuilder.activity): survive + bot.SlaveBuilder being disowned in the middle of a build + + * buildbot/status/base.py (StatusReceiverMultiService): oops, make + this inherit from StatusReceiver. Also upcall in __init__. This + fixes the embarrasing crash when the new buildSetSubmitted method + is invoked and Waterfall/etc don't implement their own. + * buildbot/test/test_run.py: add a TODO note about a test to catch + just this sort of thing. + + * buildbot/process/builder.py (Builder.attached): remove the + already-attached warning, this situation is normal. Add some + comments explaining it. + +2005-10-02 Brian Warner + + * buildbot/changes/maildir.py (Maildir.start): Tolerate + OverflowError when setting up dnotify, because some 64-bit systems + have problems with signed-vs-unsigned constants and trip up on the + DN_MULTISHOT flag. Patch from Brad Hards. + +2005-09-06 Fred Drake + + * buildbot/process/step.py (BuildStep, ShellCommand): Add + progressMetrics, description, descriptionDone to the 'parms' list, + and make use the 'parms' list from the implementation class + instead of only BuildStep to initialize the parameters. This + allows buildbot.process.factory.s() to initialize all the parms, + not just those defined in directly by BuildStep. + +2005-09-03 Brian Warner + + * NEWS: start adding items for the next release + + * docs/examples/twisted_master.cfg: (sync with reality) turn off + python2.2 tests, change 'Quick' builder to only use python2.3 + +2005-09-02 Fred Drake + + * buildbot/status/html.py (StatusResourceBuilder.body): only show + the "Ping Builder" button if the build control is available; the + user sees an exception otherwise + + * docs/buildbot.texinfo (PBChangeSource): fix a typo + +2005-09-01 Brian Warner + + * buildbot/interfaces.py (IBuilderStatus.getState): update + signature, point out that 'build' can be None + (IBuildStatus.getETA): point out ETA can be none + + * buildbot/status/html.py (CurrentBox.getBox): tolerate build/ETA + being None + * buildbot/status/words.py (IrcStatusBot.emit_status): same + +2005-08-31 Brian Warner + + * buildbot/status/base.py (StatusReceiver.builderChangedState): + update to match correct signature: removed 'eta' argument + * buildbot/status/mail.py (MailNotifier.builderChangedState): same + +2005-08-30 Brian Warner + + * buildbot/status/builder.py (LogFile): remove the assertion that + blows up when you try to overwrite an existing logfile, instead + just emit a warning. This case gets hit when the buildmaster is + killed and doesn't get a chance to write out the serialized + BuilderStatus object, so the .nextBuildNumber attribute gets out + of date. + + * buildbot/scripts/runner.py (sendchange): add --revision_file to + the 'buildbot sendchange' arguments, for the Darcs context file + * docs/buildbot.texinfo (sendchange): document it + + * buildbot/status/html.py: add pending/upcoming builds to CurrentBox + * buildbot/interfaces.py (IScheduler.getPendingBuildTimes): new method + (IStatus.getSchedulers): new method + * buildbot/status/builder.py (BuilderStatus): track pendingBuilds + (Status.getSchedulers): implement + * buildbot/process/builder.py (Builder): maintain + BuilderStatus.pendingBuilds + * buildbot/scheduler.py (Scheduler.getPendingBuildTimes): new method + (TryBase.addChange): Try schedulers should ignore Changes + + * buildbot/scripts/tryclient.py (getTopdir): implement getTopdir + for 'try' on CVS/SVN + * buildbot/test/test_runner.py (Try.testGetTopdir): test case + + * buildbot/scripts/tryclient.py (Try): make jobdir-style 'try' + report status properly. + (Try.createJob): implement unique buildset IDs + + * buildbot/status/client.py (StatusClientPerspective): add a + perspective_getBuildSets method for the benefit of jobdir-style + 'try'. + * docs/buildbot.texinfo (try): more docs + * buildbot/test/test_scheduler.py (Scheduling.testGetBuildSets): + new test case + +2005-08-18 Brian Warner + + * buildbot/scripts/tryclient.py (Try): make 'try' status reporting + actually work. It's functional but still kind of clunky. Also, it + only works with the pb-style.. needs to be made to work with the + jobdir-style too. + + * buildbot/status/client.py (RemoteBuildSet): new class + (RemoteBuildRequest): same + (RemoteBuild.remote_waitUntilFinished): return the RemoteBuild + object, not the internal BuildStatus object. + (RemoteBuild.remote_subscribe): new method to subscribe to builds + outside of the usual buildStarted() return value. + (BuildSubscriber): support class for RemoteBuild.remote_subscribe + + * buildbot/scheduler.py (Try_Jobdir): convey buildsetID properly + (Try_Userpass_Perspective.perspective_try): return a remotely + usable BuildSetStatus object + + * buildbot/interfaces.py (IBuildStatus): remove obsolete + isStarted()/waitUntilStarted() + +2005-08-16 Brian Warner + + * buildbot/status/builder.py: implement IBuildSetStatus and + IBuildRequestStatus, wire them into place. + * buildbot/buildset.py: same. Add ID, move wait-until-finished + methods into the BuildSetStatus object. + * buildbot/interfaces.py: same + (IStatus.getBuildSets): new method to get pending BuildSets + (IStatusReceiver.buildsetSubmitted): new method which hears about + new BuildSets + * buildbot/master.py (BuildMaster.submitBuildSet): same + * buildbot/process/base.py (BuildRequest): same, replace + waitUntilStarted with subscribe/unsubscribe + * buildbot/process/builder.py (BuilderControl.forceBuild): use + subscribe instead of waitUntilStarted + * buildbot/status/base.py (StatusReceiver.buildsetSubmitted): stub + for new method + * buildbot/status/client.py (StatusClientPerspective.builderRemoved): + same + * buildbot/test/test_buildreq.py: update for new code + * buildbot/test/test_control.py (Force.testRequest): same + + + * buildbot/slave/commands.py (Darcs.doVCFull): fix get-revision + for Darcs to not use the tempfile module, so it works under + python-2.2 too. We really didn't need the full cleverness of that + module, since the slave has exclusive control of its own builddir. + +2005-08-15 Brian Warner + + * buildbot/scripts/tryclient.py (CVSExtractor): implement 'try' + for CVS trees. It doesn't work for non-trunk branches, + unfortunately. + * buildbot/test/test_vc.py (CVS.testTry): test it, but skip the + branch test + + * Makefile: make it easier to test against python2.2 + + * buildbot/test/test_vc.py (VCBase.tearDown): provide for + tearDown2, so things like Arch can unregister archives as they're + shutting down. The previous subclass-override-tearDown technique + resulted in a nested maybeWait() and test failures under + Twisted-1.3.0 + + * buildbot/scripts/tryclient.py (getSourceStamp): extract branches + where we can (Arch), add a branch= argument to set the branch used + when we can't + (BazExtractor): extract the branch too + (TlaExtractor): same + * buildbot/scripts/runner.py (TryOptions): add --branch + * docs/buildbot.texinfo (try): document --branch/try_branch + + * buildbot/slave/commands.py (Darcs): implement get-revision for + Darcs, so that 'try' will work. This requires the tempfile module + from python-2.3 . + + * buildbot/test/test_vc.py: rewrite tests, getting better coverage + of revisions, branches, and 'try' in the process. + +2005-08-11 Brian Warner + + * buildbot/master.py (DebugPerspective.perspective_pokeIRC): fix + this, it got broken at some point in the last few releases + * buildbot/status/words.py (IrcBuildRequest): reply was broken + (IrcStatusBot.emit_status): handle new IBuilderStatus.getState, + specifically the removal of ETA information from the tuple + + * buildbot/locks.py: use %d for id() instead of %x, avoid a silly + warning message + + * docs/buildbot.texinfo (try): document both --builder and + 'try_builders' in .buildbot/options + * buildbot/scripts/runner.py (TryOptions): add --builder, + accumulate the values into opts['builders'] + * buildbot/scripts/tryclient.py (Try.__init__): set builders + * buildbot/test/test_runner.py (Try): add some quick tests to make + sure 'buildbot try --options' and .buildbot/options get parsed + * buildbot/test/test_scheduler.py (Scheduling.testTryUserpass): + use --builder control + + * docs/buildbot.texinfo (try): add --port argument to PB style + + * buildbot/scripts/tryclient.py (SourceStampExtractor): return an + actual SourceStamp. Still need to extract a branch name, somehow. + (Try): finish implementing the try client side, still need a UI + for specifying which builders to use + (Try.getopt): factor our options/config-file reading + * buildbot/test/test_scheduler.py (Scheduling.testTryUserpass): + test it + * buildbot/test/test_vc.py: match SourceStampExtractor change + + * buildbot/scripts/runner.py (Options.opt_verbose): --verbose + causes the twisted log to be sent to stderr + + * buildbot/scheduler.py (Try_Userpass): implement the PB style + +2005-08-10 Brian Warner + + * buildbot/scripts/runner.py: Add 'buildbot try' command, jobdir + style is 90% done, still missing status reporting or waiting for + the buildsets to finish, and it is completely untested. + + * buildbot/trybuild.py: delete file, move contents to .. + * buildbot/scripts/tryclient.py (getSourceStamp): .. here + * buildbot/test/test_vc.py: match the move + + * buildbot/scheduler.py (Try_Jobdir): implement the jobdir style + of the TryScheduler, no buildsetID or status-tracking support yet + * buildbot/test/test_scheduler.py (Scheduling.testTryJobdir): test it + + * buildbot/changes/maildir.py (Maildir.setBasedir): make it + possible to set the basedir after __init__ time, so it is easier + to use as a Service-child of the BuildMaster instance + + * buildbot/changes/maildirtwisted.py (MaildirService): make a form + that delivers messages to its Service parent instead of requiring + a subclass to be useful. This turns out to be much easier to build + unit tests around. + + * buildbot/scripts/tryclient.py (createJob): utility code to + create jobfiles, will eventually be used by 'buildbot try' + +2005-08-08 Brian Warner + + * docs/buildbot.texinfo (try): add docs on the + as-yet-unimplemented Try scheduler + + * buildbot/test/test_buildreq.py: move Scheduling tests out to .. + * buildbot/test/test_scheduler.py: .. here + (Scheduling.testTryJobdir): add placeholder test for 'try' + + * buildbot/test/test_status.py (Log.testMerge3): update to match new + addEntry merging (>=chunkSize) behavior + (Log.testConsumer): update to handle new callLater(0) behavior + + * buildbot/test/test_web.py: rearrange tests a bit, add test for + both the MAX_LENGTH bugfix and the resumeProducing hang. + + * buildbot/status/builder.py (LogFileProducer.resumeProducing): + put off the actual resumeProducing for a moment with + reactor.callLater(0). This works around a twisted-1.3.0 bug which + causes large logfiles to hang midway through. + + * buildbot/process/step.py (BuildStep.addCompleteLog): break the + logfile up into chunks, both to avoid NetstringReceiver.MAX_LENGTH + and to improve memory usage when streaming the file out to a web + browser. + * buildbot/status/builder.py (LogFile.addEntry): change > to >= to + make this work cleanly + +2005-08-03 Brian Warner + + * buildbot/trybuild.py: new file for 'try' utilities + (getSourceStamp): run in a tree, find out the baserev+patch + * buildbot/test/test_vc.py (VCBase.do_getpatch): test it, + implemented for SVN and Darcs, still working on Arch. I don't know + how to make CVS work yet. + + * docs/buildbot.texinfo: document the 'buildbot' command-line + tool, including the not-yet-implemented 'try' feature, and the + in-flux .buildbot/ options directory. + +2005-07-20 Brian Warner + + * buildbot/locks.py: added temporary id() numbers to Lock + descriptions, to track down a not-really-sharing-the-Lock bug + + * buildbot/test/runutils.py: must import errno, cut-and-paste bug + + * buildbot/test/test_slavecommand.py (ShellBase.failUnlessIn): + needed for python2.2 compatibility + * buildbot/test/test_vc.py: python2.2 compatibility: generators + are from the __future__ + +2005-07-19 Brian Warner + + * buildbot/master.py (BuildMaster.loadConfig): give a better error + message when schedulers use unknown builders + + * buildbot/process/builder.py (Builder.compareToSetup): make sure + SlaveLock('name') and MasterLock('name') are distinct + + * buildbot/master.py (BuildMaster.loadConfig): oops, sanity-check + c['schedulers'] in such a way that we can actually accept + Dependent instances + * buildbot/test/test_config.py: check it + + * buildbot/scheduler.py (Dependent.listBuilderNames): oops, add + utility method to *all* the Schedulers + (Periodic.listBuilderNames): same + + * docs/buildbot.texinfo (Interlocks): update chapter to match + reality + + * buildbot/master.py (BuildMaster.loadConfig): Add sanity checks + to make sure that c['sources'], c['schedulers'], and c['status'] + are all lists of the appropriate objects, and that the Schedulers + all point to real Builders + * buildbot/interfaces.py (IScheduler, IUpstreamScheduler): add + 'listBuilderNames' utility method to support this + * buildbot/scheduler.py: implement the utility method + * buildbot/test/test_config.py (ConfigTest.testSchedulers): test it + + * docs/buildbot.texinfo: add some @cindex entries + + * buildbot/test/test_vc.py (Arch.createRepository): set the tla ID + if it wasn't already set: most tla commands will fail unless one + has been set. + (Arch.createRepository): and disable bazaar's revision cache, since + they cause test failures (the multiple repositories we create all + interfere with each other through the cache) + + * buildbot/test/test_web.py (WebTest): remove use of deferredResult, + bring it properly up to date with twisted-2.0 test guidelines + + * buildbot/master.py (BuildMaster): remove references to old + 'interlock' module, this caused a bunch of post-merge test + failures + * buildbot/test/test_config.py: same + * buildbot/process/base.py (Build): same + + * buildbot/test/test_slaves.py: stubs for new test case + + * buildbot/scheduler.py: add test-case-name tag + * buildbot/test/test_buildreq.py: same + + * buildbot/slave/bot.py (SlaveBuilder.__init__): remove some + unnecessary init code + (Bot.remote_setBuilderList): match it + + * docs/buildbot.texinfo (@settitle): don't claim version 1.0 + + * buildbot/changes/mail.py (parseSyncmail): update comment + + * buildbot/test/test_slavecommand.py: disable Shell tests on + platforms that don't suport IReactorProcess + + * buildbot/status/builder.py (LogFile): remove the 't' mode from + all places where we open logfiles. It causes OS-X to open the file + in some weird mode that that prevents us from mixing reads and + writes to the same filehandle, which we depend upon to implement + _generateChunks properly. This change doesn't appear to break + win32, on which "b" and "t" are treated differently but a missing + flag seems to be interpreted as "t". + +2005-07-18 Brian Warner + + * buildbot/slave/commands.py (ShellCommand): overhaul + error-handling code, to try and make timeout/interrupt work + properly, and make win32 happier + * buildbot/test/test_slavecommand.py: clean up, stop using + reactor.iterate, add tests for timeout and interrupt + * buildbot/test/sleep.py: utility for a new timeout test + + * buildbot/twcompat.py: copy over twisted 1.3/2.0 compatibility + code from the local-usebranches branch + +2005-07-17 Brian Warner + + * buildbot/process/process_twisted.py + (TwistedReactorsBuildFactory): change the treeStableTimer to 5 + minutes, to match the other twisted BuildFactories, and don't + excuse failures in c/qt/win32 reactors any more. + + * docs/examples/twisted_master.cfg: turn off the 'threadless' and + 'freebsd' builders, since the buildslaves have been unavailable + for quite a while + +2005-07-13 Brian Warner + + * buildbot/test/test_vc.py (VCBase.do_branch): test the new + build-on-branch feature + + * buildbot/process/step.py (Darcs.__init__): add base_url and + default_branch arguments, just like SVN + (Arch.__init__): note that the version= argument is really the + default branch name + + * buildbot/slave/commands.py (SourceBase): keep track of the + repository+branch that was used for the last checkout in + SRCDIR/.buildbot-sourcedata . If the contents of this file do not + match, we clobber the directory and perform a fresh checkout + rather than trying to do an in-place update. This should protect + us against trying to get to branch B by doing an update in a tree + obtained from branch A. + (CVS.setup): add CVS-specific sourcedata: root, module, and branch + (SVN.setup): same, just the svnurl + (Darcs.setup): same, just the repourl + (Arch.setup): same, arch coordinates (url), version, and + buildconfig. Also pull the buildconfig from the args dictionary, + which we weren't doing before, so the build-config was effectively + disabled. + (Arch.sourcedirIsUpdateable): don't try to update when we're + moving to a specific revision: arch can't go backwards, so it is + safer to just clobber the tree and checkout a new one at the + desired revision. + (Bazaar.setup): same sourcedata as Arch + + * buildbot/test/test_dependencies.py (Dependencies.testRun_Fail): + use maybeWait, to work with twisted-1.3.0 and twcompat + (Dependencies.testRun_Pass): same + + * buildbot/test/test_vc.py: rearrange, cleanup + + * buildbot/twcompat.py: add defer.waitForDeferred and + utils.getProcessOutputAndValue, so test_vc.py (which uses them) + can work under twisted-1.3.0 . + + * buildbot/test/test_vc.py: rewrite. The sample repositories are + now created at setUp time. This increases the runtime of the test + suite considerably (from 91 seconds to 151), but it removes the + need for an offline tarball, which should solve a problem I've + seen where the test host has a different version of svn than the + tarball build host. The new code also validates that mode=update + really picks up recent commits. This approach will also make it + easier to test out branches, because the code which creates the VC + branches is next to the code which uses them. It will also make it + possible to test some change-notification hooks, by actually + performing a VC commit and watching to see the ChangeSource get + notified. + +2005-07-12 Brian Warner + + * docs/buildbot.texinfo (SVN): add branches example + * docs/Makefile (buildbot.ps): add target for postscript manual + + * buildbot/test/test_dependencies.py: s/test_interlocks/test_locks/ + * buildbot/test/test_locks.py: same + + * buildbot/process/step.py (Darcs): comment about default branches + + * buildbot/master.py (BuildMaster.loadConfig): don't look for + c['interlocks'] in the config file, complain if it is present. + Scan all locks in c['builders'] to make sure the Locks they use + are uniquely named. + * buildbot/test/test_config.py: remove old c['interlocks'] test, + add some tests to check for non-uniquely-named Locks + * buildbot/test/test_vc.py (Patch.doPatch): fix factory.steps, + since the unique-Lock validation code requires it now + + * buildbot/locks.py: fix test-case-name + + * buildbot/interlock.py: remove old file + +2005-07-11 Brian Warner + + * buildbot/test/test_interlock.py: rename to.. + * buildbot/test/test_locks.py: .. something shorter + + * buildbot/slave/bot.py (BuildSlave.stopService): newer Twisted + versions (after 2.0.1) changed internet.TCPClient to shut down the + connection in stopService. Change the code to handle this + gracefully. + + * buildbot/process/base.py (Build): handle whole-Build locks + * buildbot/process/builder.py (Builder.compareToSetup): same + * buildbot/test/test_interlock.py: make tests work + + * buildbot/process/step.py (BuildStep.startStep): complain if a + Step tries to claim a lock that's owned by its own Build + (BuildStep.releaseLocks): typo + + * buildbot/locks.py (MasterLock): use ComparableMixin so config + file reloads don't replace unchanged Builders + (SlaveLock): same + * buildbot/test/test_config.py (ConfigTest.testInterlocks): + rewrite to cover new Locks instead of old c['interlocks'] + * buildbot/test/runutils.py (RunMixin.connectSlaves): remember + slave2 too + + + * buildbot/test/test_dependencies.py (Dependencies.setUp): always + start the master and connect the buildslave + + * buildbot/process/step.py (FailingDummy.done): finish with a + FAILURE status rather than raising an exception + + * buildbot/process/base.py (BuildRequest.mergeReasons): don't try to + stringify a BuildRequest.reason that is None + + * buildbot/scheduler.py (BaseUpstreamScheduler.buildSetFinished): + minor fix + * buildbot/status/builder.py (BuildSetStatus): implement enough to + allow scheduler.Dependent to work + * buildbot/buildset.py (BuildSet): set .reason and .results + + * buildbot/test/test_interlock.py (Locks.setUp): connect both + slaves, to make the test stop hanging. It still fails, of course, + because I haven't even started to implement Locks. + + * buildbot/test/runutils.py (RunMixin.connectSlaves): new utility + + * docs/buildbot.texinfo (Build-Dependencies): redesign the feature + * buildbot/interfaces.py (IUpstreamScheduler): new Interface + * buildbot/scheduler.py (BaseScheduler): factor out common stuff + (Dependent): new class for downstream build dependencies + * buildbot/test/test_dependencies.py: tests (still failing) + + * buildbot/buildset.py (BuildSet.waitUntilSuccess): minor notes + +2005-07-07 Brian Warner + + * buildbot/test/runutils.py (RunMixin): factored this class out.. + * buildbot/test/test_run.py: .. from here + * buildbot/test/test_interlock.py: removed old c['interlock'] tests, + added new buildbot.locks tests (which all hang right now) + * buildbot/locks.py (SlaveLock, MasterLock): implement Locks + * buildbot/process/step.py: claim/release per-BuildStep locks + + * docs/Makefile: add 'buildbot.html' target + + * buildbot/process/step.py (CVS.__init__): allow branch=None to be + interpreted as "HEAD", so that all VC steps can accept branch=None + and have it mean the "default branch". + + * docs/buildbot.texinfo: add Schedulers, Dependencies, and Locks + +2005-07-07 Brian Warner + + * docs/examples/twisted_master.cfg: update to match current usage + + * docs/buildbot.texinfo (System Architecture): comment out the + image, it doesn't exist yet and just screws up the HTML manual. + +2005-07-05 Brian Warner + + * debian/.cvsignore: oops, missed one. Removing leftover file. + +2005-06-17 Brian Warner + + * buildbot/test/test_vc.py (VCSupport.__init__): svn --version + changed its output in 1.2.0, don't mistakenly think that the + subversion we find isn't capable of supporting our tests. + + * debian/*: remove the debian/ directory and its contents, to make + life easier for the proper Debian maintainer + * MANIFEST.in: same + * Makefile (release): same + +2005-06-07 Brian Warner + + * everything: create a distinct SourceStamp class to replace the + ungainly 4-tuple, let it handle merging instead of BuildRequest. + Changed the signature of Source.startVC to include the revision + information (instead of passing it through self.args). Implement + branches for SVN (now only Darcs/Git is missing support). Add more + Scheduler tests. + +2005-06-06 Brian Warner + + * everything: rearrange build scheduling. Create a new Scheduler + object (configured in c['schedulers'], which submit BuildSets to a + set of Builders. Builders can now use multiple slaves. Builds can + be run on alternate branches, either requested manually or driven + by changes. This changed some of the Status classes. Interlocks + are out of service until they've been properly split into Locks + and Dependencies. treeStableTimer, isFileImportant, and + periodicBuild have all been moved from the Builder to the + Scheduler. + (BuilderStatus.currentBigState): removed the 'waiting' and + 'interlocked' states, removed the 'ETA' argument. + +2005-05-24 Brian Warner + + * buildbot/pbutil.py (ReconnectingPBClientFactory): Twisted-1.3 + erroneously abandons the connection (in clientConnectionFailed) + for non-UserErrors, which means that if we lose the connection due + to a network problem or a timeout, we'll never try to reconnect. + Fix this by not upcalling to the buggy parent method. Note: + twisted-2.0 fixes this, but the function only has 3 lines so it + makes more sense to copy it than to try and detect the buggyness + of the parent class. Fixes SF#1207588. + + * buildbot/changes/changes.py (Change.branch): doh! Add a + class-level attribute to accomodate old Change instances that were + pickled before 0.6.5 (where .branch was added for new Changes). + This fixes the exception that occurs when you try to look at an + old Change (through asHTML). + + * buildbot/__init__.py (version): bump to 0.6.6+ while between + releases + 2005-05-23 Brian Warner * buildbot/__init__.py (version): release 0.6.6 Index: NEWS =================================================================== RCS file: /cvsroot/buildbot/site/NEWS,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- NEWS 23 May 2005 23:53:55 -0000 1.8 +++ NEWS 25 Oct 2005 03:53:20 -0000 1.9 @@ -1,5 +1,113 @@ User visible changes in Buildbot. +* Release 0.7.0 (24 Oct 2005) + +** new features + +*** new c['schedulers'] config-file element (REQUIRED) + +The code which decides exactly *when* a build is performed has been massively +refactored, enabling much more flexible build scheduling. YOU MUST UPDATE +your master.cfg files to match: in general this will merely require you to +add an appropriate c['schedulers'] entry. Any old ".treeStableTime" settings +on the BuildFactory instances will now be ignored. The user's manual has +complete details with examples of how the new Scheduler classes work. + +*** c['interlocks'] removed, Locks and Dependencies now separate items + +The c['interlocks'] config element has been removed, and its functionality +replaced with two separate objects. Locks are used to tell the buildmaster +that certain Steps or Builds should not run at the same time as other Steps +or Builds (useful for test suites that require exclusive access to some +external resource: of course the real fix is to fix the tests, because +otherwise your developers will be suffering from the same limitations). The +Lock object is created in the config file and then referenced by a Step +specification tuple or by the 'locks' key of the Builder specification +dictionary. Locks come in two flavors: MasterLocks are buildmaster-wide, +while SlaveLocks are specific to a single buildslave. + +When you want to have one Build run or not run depending upon whether some +other set of Builds have passed or failed, you use a special kind of +Scheduler defined in the scheduler.Dependent class. This scheduler watches an +upstream Scheduler for builds of a given source version to complete, and only +fires off its own Builders when all of the upstream's Builders have built +that version successfully. + +Both features are fully documented in the user's manual. + +*** 'buildbot try' + +The 'try' feature has finally been added. There is some configuration +involved, both in the buildmaster config and on the developer's side, but +once in place this allows the developer to type 'buildbot try' in their +locally-modified tree and to be given a report of what would happen if their +changes were to be committed. This works by computing a (base revision, +patch) tuple that describes the developer's tree, sending that to the +buildmaster, then running a build with that source on a given set of +Builders. The 'buildbot try' tool then emits status messages until the builds +have finished. + +'try' exists to allow developers to run cross-platform tests on their code +before committing it, reducing the chances they will inconvenience other +developers by breaking the build. The UI is still clunky, but expect it to +change and improve over the next few releases. + +Instructions for developers who want to use 'try' (and the configuration +changes necessary to enable its use) are in the user's manual. + +*** Build-On-Branch + +When suitably configured, the buildbot can be used to build trees from a +variety of related branches. You can set up Schedulers to build a tree using +whichever branch was last changed, or users can request builds of specific +branches through IRC, the web page, or (eventually) the CLI 'buildbot force' +subcommand. + +The IRC 'force' command now takes --branch and --revision arguments (not that +they always make sense). Likewise the HTML 'force build' button now has an +input field for branch and revision. Your build's source-checkout step must +be suitably configured to support this: for SVN it involves giving both a +base URL and a default branch. Other VC systems are configured differently. +The ChangeSource must also provide branch information: the 'buildbot +sendchange' command now takes a --branch argument to help hook script writers +accomplish this. + +*** Multiple slaves per Builder + +You can now attach multiple buildslaves to each Builder. This can provide +redundancy or primitive load-balancing among many machines equally capable of +running the build. To use this, define a key in the Builder specification +dictionary named 'slavenames' with a list of buildslave names (instead of the +usual 'slavename' that contains just a single slavename). + +*** minor new features + +The IRC and email status-reporting facilities now provide more specific URLs +for particular builds, in addition to the generic buildmaster home page. The +HTML per-build page now has more information. + +The Twisted-specific test classes have been modified to match the argument +syntax preferred by Trial as of Twisted-2.1.0 and newer. The generic trial +steps are still suitable for the Trial that comes with older versions of +Twisted, but may produce deprecation warnings or errors when used with the +latest Trial. + +** bugs fixed + +DNotify, used by the maildir-watching ChangeSources, had problems on some +64-bit systems relating to signed-vs-unsigned constants and the DN_MULTISHOT +flag. A workaround was provided by Brad Hards. + +The web status page should now be valid XHTML, thanks to a patch by Brad +Hards. The charset parameter is specified to be UTF-8, so VC comments, +builder names, etc, should probably all be in UTF-8 to be displayed properly. + +** creeping version dependencies + +The IRC 'force build' command now requires python2.3 (for the shlex.split +function). + + * Release 0.6.6 (23 May 2005) ** bugs fixed Index: README =================================================================== RCS file: /cvsroot/buildbot/site/README,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- README 18 May 2005 08:30:48 -0000 1.14 +++ README 25 Oct 2005 03:53:20 -0000 1.15 @@ -56,19 +56,20 @@ available in python-2.1, and both master and slave require a version of Twisted which only works with python-2.2 or later. Certain features (like the inclusion of build logs in status emails) require python-2.2.2 or - later. + later, while the IRC 'force' command requires python-2.3 . Twisted: http://twistedmatrix.com Both the buildmaster and the buildslaves require Twisted-1.3.0 or later. - They have been briefly tested against Twisted-1.2.0, and might even work - with Twisted-1.1.0, but 1.3.0 is the version that has received the most - testing. + It has been mainly developed against Twisted-2.0.1, but has been tested + against Twisted-2.1.0 (the most recent as this time), and might even work + on versions as old as Twisted-1.1.0, but as always the most recent version + is recommended. - Both work with Twisted-2.0.0 as well. You'll need at least "Twisted" (the - core package), and you'll also want TwistedMail, TwistedWeb, and - TwistedWords (for sending email, serving a web status page, and delivering - build status via IRC, respectively). + When using the split subpackages of Twisted-2.x.x, you'll need at least + "Twisted" (the core package), and you'll also want TwistedMail, + TwistedWeb, and TwistedWords (for sending email, serving a web status + page, and delivering build status via IRC, respectively). CVSToys: http://purl.net/net/CVSToys @@ -80,8 +81,9 @@ INSTALLATION: -Please read the User's Manual in docs/buildbot.info for complete -instructions. +Please read the User's Manual in docs/buildbot.info (or in HTML form on the +buildbot web site) for complete instructions. This file only contains a brief +summary. RUNNING THE UNIT TESTS @@ -89,20 +91,12 @@ PYTHONPATH=. trial -v buildbot.test -This should run up to 124 tests, depending upon what VC tools you have -installed. On my desktop machine it takes about two minutes to complete. +This should run up to 175 tests, depending upon what VC tools you have +installed. On my desktop machine it takes about four minutes to complete. Nothing should fail, a few might be skipped. If any of the tests fail, you should stop and investigate the cause before continuing the installation process, as it will probably be easier to track down the bug early. -If you want to test the VC checkout process, you'll need to install a -tarball of repositories, available from http://buildbot.sf.net/ . Otherwise -there are about 8 tests which will be skipped (all with names like testSVN -and testArchHTTP). If you unpack this tarball in ~/tmp, it will create -~/tmp/buildbot-test-vc-1, and you can enable the extra tests with: - - PYTHONPATH=. BUILDBOT_TEST_VC=~/tmp trial -v buildbot.test - INSTALLING THE LIBRARIES: @@ -138,13 +132,14 @@ buildbot slave WORKDIR MASTERHOST:PORT SLAVENAME PASSWORD -This will create a "TAP" file called "buildbot.tap", which bundles up all -the state needed by the build slave application. Twisted has a tool called -"twistd" which knows how to load these saved applications and start running -them. twistd takes care of logging and daemonization (running the program in -the background). /usr/bin/buildbot is a front end which runs twistd for you. +This will create a file called "buildbot.tac", which bundles up all the state +needed by the build slave application. Twisted has a tool called "twistd" +which knows how to load these saved applications and start running them. +twistd takes care of logging and daemonization (running the program in the +background). /usr/bin/buildbot is a front end which runs twistd for you. -Once you have the .tap file, you start it running like this: +Once you've set up the directory with the .tac file, you start it running +like this: buildbot start WORKDIR @@ -180,65 +175,14 @@ SETTING UP A BUILD MASTER: -First, read through the rest of the documents to understand the definition -of Builds and BuildFactories. Look at the docs in docs/*.xhtml. - -Second, know that this should get easier in the future, probably with a -web-based interface to create new kinds of Builds and manipulate them. - -You start by picking a base directory where the buildmaster will store all -its status and logfiles. Then you create the buildmaster's buildbot.tap file -with the 'buildbot' tool: - - buildbot master WORKDIR - -This will also create a sample configuration file for you in -WORKDIR/master.cfg . Edit this to describe how your buildmaster should -operate: what port it should listen on for connections from the build slaves, -which build slaves are allowed to connect, how to watch for changes to the -source tree, how to run builds, and how to deliver status information. - -There are more examples in docs/examples/, and plenty of documentation on the -various settings in docs/ . Everything is controlled by the config file. It -must be readable from the master's basedir under the filename 'master.cfg'. - -Then you launch the buildmaster just as you would launch the buildslaves: the -'buildbot' tool runs twistd, which automatically forks off the daemon into -the background: - - buildbot start WORKDIR - -All buildmaster output is logged into 'twistd.log' in that same directory. - -The buildmaster will read master.cfg, set up the builders, start listening -on the web server port and begin accepting connections from the buildslaves. - - - CONNECTING TO A FRESHCVS DAEMON: - -Set up CVSToys-1.0.10, and add a statement like the following to your -freshCfg file: - -pb = ConfigurationSet([ - (None, None, None, PBService(userpass=('foo', 'bar'), port=4519)), - ]) - -This will announce all changes to a client which connects to port 4519 using -a username of 'foo' and a password of 'bar'. - -Then add a clause like this to your buildmaster's master.cfg: - -BuildmasterConfig['sources'] = [FreshCVSSource("cvs.example.com", 4519, - "foo", "bar", - prefix="glib/")] +Please read the user's manual for instructions. The short form is that you +use 'buildbot master MASTERDIR' to create the base directory, then you edit +the 'master.cfg' file to configure the buildmaster. Once this is ready, you +use 'buildbot START MASTERDIR' to launch it. -where "cvs.example.com" is the host that is running the FreshCVS daemon, and -"glib" is the top-level directory (relative to the repository's root) where -all your source code lives. Most projects keep one or more projects in the -same repository (along with CVSROOT/ to hold admin files like loginfo and -freshCfg); the prefix= argument tells the buildmaster to ignore everything -outside that directory, and to strip that common prefix from all pathnames -it handles. +A sample configuration file will be created for you in WORKDIR/master.cfg . +There are more examples in docs/examples/, and plenty of documentation in the +user's manual. Everything is controlled by the config file. SUPPORT: Index: index.html =================================================================== RCS file: /cvsroot/buildbot/site/index.html,v retrieving revision 1.48 retrieving revision 1.49 diff -u -d -r1.48 -r1.49 --- index.html 28 Sep 2005 07:41:06 -0000 1.48 +++ index.html 25 Oct 2005 03:53:20 -0000 1.49 @@ -31,7 +31,7 @@

      Current contents:

      @@ -177,5 +173,5 @@ align="right" /> -Last modified: Wed Sep 28 00:39:19 PDT 2005 +Last modified: Mon Oct 24 20:51:19 PDT 2005 --- NEW FILE: manual-0.7.0.html --- BuildBot Manual 0.7.0