[Buildbot-commits] site ChangeLog,1.17,1.18 NEWS,1.3,1.4 index.html,1.33,1.34

Brian Warner warner at users.sourceforge.net
Mon Dec 13 08:39:38 UTC 2004


Update of /cvsroot/buildbot/site
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18329

Modified Files:
	ChangeLog NEWS index.html 
Log Message:
update for 0.6.2, add spamassassin.org buildbot

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/site/ChangeLog,v
retrieving revision 1.17
retrieving revision 1.18
diff -u -d -r1.17 -r1.18
--- ChangeLog	23 Nov 2004 11:41:00 -0000	1.17
+++ ChangeLog	13 Dec 2004 08:39:34 -0000	1.18
@@ -1,3 +1,289 @@
+2004-12-13  Brian Warner  <warner at lothar.com>
+
+	* buildbot/__init__.py (version): Releasing buildbot-0.6.2
+
+	* debian/changelog: update for 0.6.2
+	* NEWS: finalize for 0.6.2
+
+2004-12-11  Brian Warner  <warner at lothar.com>
+
+	* NEWS: bring it up to date
+
+	* buildbot/slave/bot.py (BotFactory): revamp keepalive/lost-master
+	detection code. Require some sign of life from the buildmaster
+	every BotFactory.keepaliveInterval seconds. Provoke this
+	indication at BotFactory.keepaliveTimeout seconds before the
+	deadline by sending a keepalive request. We don't actually care if
+	that request is answered in a timely fashion, what we care about
+	is that .activity() is called before the deadline. .activity() is
+	triggered by any PB message from the master (including an ack to
+	one of the slave's status-update messages). With this new scheme,
+	large status messages over slow pipes are OK, as long as any given
+	message can be sent (and thus acked) within .keepaliveTimeout
+	seconds (which defaults to 30).
+	(SlaveBuilder.remote_startCommand): record activity
+	(SlaveBuilder.ackUpdate): same
+	(SlaveBuilder.ackComplete): same
+	(BotFactory.gotPerspective): same
+	* buildbot/test/test_run.py (Disconnect.testSlaveTimeout): test it
+
+2004-12-09  Brian Warner  <warner at lothar.com>
+
+	* buildbot/status/html.py (StatusResourceBuilder.getChild): remove
+	debug message
+
+	* buildbot/process/step_twisted.py (Trial._commandComplete):
+	update self.cmd when we start the 'cat test.log' transfer. Without
+	this, we cannot interrupt the correct RemoteCommand when we lose
+	the connection.
+
+	* buildbot/process/step.py (RemoteCommand.interrupt): don't bother
+	trying to tell the slave to stop the command if we're already
+	inactive, or if we no longer have a .remote
+
+	* buildbot/process/builder.py (Builder._detached): don't let an
+	exception in currentBuild.stopBuild() prevent the builder from
+	being marked offline
+
+2004-12-07  Brian Warner  <warner at lothar.com>
+
+	* buildbot/status/words.py (IrcStatusBot.getBuilder): catch the
+	KeyError that happens when you ask for a non-existent Builder, and
+	translate it into a UsageError.
+
+	* buildbot/test/test_run.py (Disconnect.testBuild4): validate that
+	losing the slave in the middle of a remote step is handled too
+
+	* buildbot/process/step.py (ShellCommand.interrupt): 'reason' can
+	be a Failure, so be sure to stringify it before using it as the
+	contents of the 'interrupt' logfile
+	(RemoteCommand.interrupt): use stringified 'why' in
+	remote_interruptCommand too, just in case
+
+2004-12-06  Brian Warner  <warner at lothar.com>
+
+	* buildbot/slave/commands.py (Arch.doVCUpdate): use 'tla replay'
+	instead of 'tla update', which is more efficient in case we've
+	missed a couple of patches since the last update.
+
+	* debian/changelog: update for previous (0.6.1) release. Obviously
+	this needs to be handled better.
+
+2004-12-05  Brian Warner  <warner at lothar.com>
+
+	* NEWS: update for stuff since last release
+
+	* buildbot/master.py (DebugPerspective.attached): return 'self', to
+	match the maybeDeferred change in Dispatcher.requestAvatar
+	* buildbot/changes/pb.py (ChangePerspective.attached): same
+	* buildbot/status/client.py (StatusClientPerspective.attached): same
+	* buildbot/process/builder.py (Builder._attached3): same
+	* buildbot/pbutil.py (NewCredPerspective.attached): same
+
+	* buildbot/status/html.py (WaterfallStatusResource.phase2): Add
+	the date to the top-most box, if it is not the same as today's
+	date.
+
+	* docs/slave.xhtml: provide a buildslave setup checklist
+
+	* docs/source.xhtml (Arch): correct terminology
+
+2004-12-04  Brian Warner  <warner at lothar.com>
+
+	* buildbot/test/test_slavecommand.py: use sys.executable instead
+	of hard-coding 'python' for child commands, might help portability
+
+	* docs/examples/twisted_master.cfg: update to current usage
+
+	* buildbot/status/words.py (IrcStatusBot.command_STOP): add a
+	'stop build' command to the IRC bot
+
+	* buildbot/master.py (Dispatcher.requestAvatar): remove debug
+	message that broke PBChangeSource
+
+	* buildbot/slave/bot.py: clean up shutdown/lose-master code
+	(SlaveBuilder): make some attributes class-level, remove the old
+	"update queue" which existed to support resuming a build after the
+	master connection was lost. Try to reimplement that feature later.
+	(SlaveBuilder.stopCommand): clear self.command when the
+	SlaveCommand finishes, so that we don't try to kill a leftover one
+	at shutdown time.
+	(SlaveBuilder.commandComplete): same, merge with commandFailed and
+	.finishCommand
+
+	* buildbot/slave/commands.py (SourceBase): set self.command for
+	all VC commands, so they can be interrupted.
+
+2004-12-03  Brian Warner  <warner at lothar.com>
+
+	* buildbot/master.py: clean up slave-handling code, to handle
+	slave-disconnect and multiple-connect better
+	(BotPerspective): make these long-lasting, exactly one per bot
+	listed in the config file.
+	(BotPerspective.attached): if a slave connects while an existing
+	one appears to still be connected, disconnect the old one first.
+	(BotPerspective.disconnect): new method to forcibly disconnect a
+	buildslave. Use some hacks to empty the transmit buffer quickly to
+	avoid the long (20-min?) TCP timeout that could occur if the old
+	slave has dropped off the net.
+	(BotMaster): Keep persistent BotPerspectives in .slaves, let them
+	own their own SlaveStatus objects. Remove .attached/.detached, add
+	.addSlave/.removeSlave, treat slaves like Builders (config file
+	parsing sends deltas to the BotMaster). Inform the slave
+	instances, i.e. the BotPerspective, about addBuilder and
+	removeBuilder.
+	(BotMaster.getPerspective): turns into a single dict lookup
+	(Dispatcher.requestAvatar): allow .attached to return a Deferred,
+	which gives BotPerspective.attached a chance to disconnect the old
+	slave first.
+	(BuildMaster.loadConfig): add code (disabled) to validate that all
+	builders use known slaves (listed in c['bots']). The check won't
+	work with tuple-specified builders, which are deprecated but not
+	yet invalid, so the check is disabled for now.
+	(BuildMaster.loadConfig_Slaves): move slave-config into a separate
+	routine, do the add/changed/removed dance with them like we do
+	with builders.
+	(BuildMaster.loadConfig_Sources): move source-config into a
+	separate routine too
+
+	* buildbot/status/builder.py (Status.getSlave): get the
+	SlaveStatus object from the BotPerspective, not the BotMaster.
+
+	* buildbot/test/test_run.py: bunch of new tests for losing the
+	buildslave at various points in the build, handling a slave that
+	connects multiple times, and making sure we can interrupt a
+	running build
+
+	* buildbot/slave/bot.py (BuildSlave): make it possible to use
+	something other than 'Bot' for the Bot object, to make certain
+	test cases easier to write.
+	(BuildSlave.waitUntilDisconnected): utility method for testing
+
+2004-11-30  Brian Warner  <warner at lothar.com>
+
+	* buildbot/test/test_run.py (RunMixin): refactor, remove debug msg
+
+	* buildbot/interfaces.py (IBuilderControl.ping): add timeout=
+	argument, return a Deferred that always fires with True or False.
+	I don't use an errback to indicate 'ping failed' so that callers
+	are free to ignore the deferred without causing spurious errors in
+	the logs.
+	* buildbot/process/builder.py (BuilderControl.ping): implement it
+
+	* buildbot/test/test_run.py (Status.testDisappear): test ping
+	(Status.disappearSlave): fix it
+
+2004-11-30  Brian Warner  <warner at lothar.com>
+
+	* buildbot/interfaces.py (IBuildControl): add .stopBuild
+	(IBuilderControl): add .getBuild(num), only works for the current
+	build, of course, although it might be interesting to offer
+	something for builds in the .waiting or .interlocked state.
+
+	* buildbot/process/base.py (Build): have .stopBuild just do the
+	interrupt, then let the build die by itself.
+	(BuildControl): add .stopBuild, and add a point-event named
+	'interrupt' just after the build so status viewers can tell that
+	someone killed it.
+	(BuilderControl): add .getBuild
+
+	* buildbot/process/step.py (Dummy): use haltOnFailure so it really
+	stops when you kill it, good for testing
+	(ShellCommand.interrupt): add a logfile named 'interrupt' which
+	contains the 'reason' text.
+
+	* buildbot/status/html.py: Add Stop Build button, if the build can
+	still be stopped. Send a Redirect (to the top page) one second
+	later, hopefully long enough for the interrupt to have an effect.
+	Move make_row() up to top-level to share it between Stop Build and
+	Force Build.
+
+	* buildbot/slave/commands.py: only kill the child process once
+
+	* buildbot/test/test_run.py: add testInterrupt
+
+2004-11-29  Brian Warner  <warner at lothar.com>
+
+	* buildbot/process/base.py: Refactor command interruption. The
+	Build is now responsible for noticing that the slave has gone
+	away: Build.lostRemote() interrupts the current step and makes
+	sure that no further ones will be started.
+	
+	* buildbot/process/builder.py: When the initial remote_startBuild
+	message fails, log it: this usually indicates that the slave has
+	gone away, but we don't really start paying attention until they
+	fail to respond to the first step's command.
+
+	* buildbot/process/step.py (RemoteCommand): Does *not* watch for
+	slave disconnect. Now sports a new interrupt() method. Error
+	handling was simplified a lot by chaining deferreds, so
+	remoteFailed/remoteComplete were merged into a single
+	remoteComplete method (which can now get a Failure object).
+	Likewise failed/finished were merged into just _finished.
+	(BuildStep): Add interrupt(why) method, and if why is a
+	ConnectionLost Failure then the step is failed with some useful
+	error text.
+
+	* buildbot/slave/bot.py: stop the current command when the remote
+	Step reference is lost, and when the slave is shut down.
+	(Bot): make it a MultiService, so it can have children. Use
+	stopService to tell when the slave is shutting down.
+	(SlaveBuilder): make it a Service, and a child of the Bot. Add
+	remote_interruptCommand (which asks the current SlaveCommand to
+	stop but allows it to keep emitting status messages), and
+	stopCommand (which tells it to shut up and die).
+
+	* buildbot/slave/commands.py: make commands interruptible
+	(ShellCommand.kill): factor out os.kill logic
+	(Command): factor out setup()
+	(Command.sendStatus): don't send status if .running is false, this
+	happens when the command has been halted.
+	(Command.interrupt): new method, used to tell the command to die
+	(SlaveShellCommand): implement .interrupt
+	(DummyCommand): implement .interrupt
+	(SourceBase, etc): factor out setup(), don't continue substeps if
+	.interrupted is set
+
+	* buildbot/status/builder.py: fix all waitUntilFinished() methods
+	so they can be called after finishing
+
+	* buildbot/test/test_run.py: new tests for disconnect behavior,
+	refactor slave-shutdown routines, add different kinds of
+	slave-shutdown
+
+2004-11-27  Brian Warner  <warner at lothar.com>
+
+	* buildbot/status/words.py (IrcStatusBot.convertTime): utility
+	method to express ETA time like "2m45s" instead of "165 seconds"
+
+2004-11-24  Brian Warner  <warner at lothar.com>
+
+	* buildbot/test/test_vc.py (VC.testArch): unregister the test
+	archive after the test completes, to avoid cluttering the user's
+	'tla archives' listing with a bogus entry. Arch doesn't happen to
+	provide any way to override the use of ~/.arch-params/, so there
+	isn't a convenient way to avoid touching the setup of the user who
+	runs the test.
+	(VC_HTTP.testArchHTTP): same
+
+2004-11-23  Brian Warner  <warner at lothar.com>
+
+	* buildbot/status/html.py (TextLog): split render() up into
+	render_HEAD and render_GET. Use a Producer when sending log
+	chunks, to reduce memory requirements and avoid sending huge
+	non-Banana-able strings over web.distrib connections. Requires
+	peeking under the covers of IStatusLog.
+	(TextLog.resumeProducing): fix the "as text" link, handle client
+	disconnects that occur while we're still sending old chunks.
+
+	* buildbot/status/builder.py (HTMLLogFile.waitUntilFinished): oops,
+	use defer.succeed, not the non-existent defer.success
+	(LogFile.waitUntilFinished): same
+	(LogFile.subscribe): don't add watchers to a finished logfile
+
+	* buildbot/__init__.py (version): bump to 0.6.1+ while between
+	releases
+
 2004-11-23  Brian Warner  <warner at lothar.com>
 
 	* buildbot/__init__.py (version): Releasing buildbot-0.6.1

Index: NEWS
===================================================================
RCS file: /cvsroot/buildbot/site/NEWS,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -d -r1.3 -r1.4
--- NEWS	23 Nov 2004 11:41:00 -0000	1.3
+++ NEWS	13 Dec 2004 08:39:35 -0000	1.4
@@ -1,5 +1,71 @@
 User visible changes in Buildbot.
 
+* Release 0.6.2 (13 Dec 2004)
+
+** new features
+
+It is now possible to interrupt a running build. Both the web page and the
+IRC bot feature 'stop build' commands, which can be used to interrupt the
+current BuildStep and accelerate the termination of the overall Build. The
+status reporting for these still leaves something to be desired (an
+'interrupt' event is pushed into the column, and the reason for the interrupt
+is added to a pseudo-logfile for the step that was stopped, but if you only
+look at the top-level status it appears that the build failed on its own).
+
+Builds are also halted if the connection to the buildslave is lost. On the
+slave side, any active commands are halted if the connection to the
+buildmaster is lost.
+
+** minor new features
+
+The IRC log bot now reports ETA times in a MMSS format like "2m45s" instead
+of the clunky "165 seconds".
+
+** bug fixes
+
+*** Slave Disconnect
+
+Slave disconnects should be handled better now: the current build should be
+abandoned properly. Earlier versions could get into weird states where the
+build failed to finish, clogging the builder forever (or at least until the
+buildmaster was restarted).
+
+In addition, there are weird network conditions which could cause a
+buildslave to attempt to connect twice to the same buildmaster. This can
+happen when the slave is sending large logfiles over a slow link, while using
+short keepalive timeouts. The buildmaster has been fixed to allow the second
+connection attempt to take precedence over the first, so that the older
+connection is jettisoned to make way for the newer one.
+
+In addition, the buildslave has been fixed to be less twitchy about timeouts.
+There are now two parameters: keepaliveInterval (which is controlled by the
+mktap 'keepalive' argument), and keepaliveTimeout (which requires editing the
+.py source to change from the default of 30 seconds). The slave expects to
+see *something* from the master at least once every keepaliveInterval
+seconds, and will try to provoke a response (by sending a keepalive request)
+'keepaliveTimeout' seconds before the end of this interval just in case there
+was no regular traffic. Any kind of traffic will qualify, including
+acknowledgements of normal build-status updates.
+
+The net result is that, as long as any given PB message can be sent over the
+wire in less than 'keepaliveTimeout' seconds, the slave should not mistakenly
+disconnect because of a timeout. There will be traffic on the wire at least
+every 'keepaliveInterval' seconds, which is what you want to pay attention to
+if you're trying to keep an intervening NAT box from dropping what it thinks
+is an abandoned connection. A quiet loss of connection will be detected
+within 'keepaliveInterval' seconds.
+
+*** Large Logfiles
+
+The web page rendering code has been fixed to deliver large logfiles in
+pieces, using a producer/consumer apparatus. This avoids the large spike in
+memory consumption when the log file body was linearized into a single string
+and then buffered in the socket's application-side transmit buffer. This
+should also avoid the 640k single-string limit for web.distrib servers that
+could be hit by large (>640k) logfiles.
+
+
+
 * Release 0.6.1 (23 Nov 2004)
 
 ** win32 improvements/bugfixes

Index: index.html
===================================================================
RCS file: /cvsroot/buildbot/site/index.html,v
retrieving revision 1.33
retrieving revision 1.34
diff -u -d -r1.33 -r1.34
--- index.html	6 Dec 2004 07:07:51 -0000	1.33
+++ index.html	13 Dec 2004 08:39:35 -0000	1.34
@@ -11,7 +11,7 @@
 Current contents:
 
 <ul>
-  <li>The current release is buildbot-0.6.1 . You can download the source
+  <li>The current release is buildbot-0.6.2 . You can download the source
   from the sf.net download page <a
   href="http://sourceforge.net/project/showfiles.php?group_id=73177">here</a>.
   The release is signed with my GPG public key, available <a
@@ -91,7 +91,10 @@
   href="http://www.mono-project.com/">Mono</a> project has a <a
   href="http://mono.ximian.com:8008/">buildbot</a> running to test the HEAD
   and release branches of the main project on several architectures.</li>
-  
+
+  <li>Justin Mason reports that the SpamAssassin project is running a <a
+  href="http://bugzilla.SpamAssassin.org:8010/">buildbot</a> too.</li>
+
   <li><b>install a Buildbot today and get your name added here!</b></li>
 </ul>
 
@@ -114,5 +117,5 @@
      align="right" />
 </a>
 
-<!-- hhmts start -->Last modified: Sun Dec  5 23:07:10 PST 2004 <!-- hhmts end -->
+<!-- hhmts start -->Last modified: Mon Dec 13 00:38:07 PST 2004 <!-- hhmts end -->
 </body> </html>





More information about the Commits mailing list