[Buildbot-devel] PB unsubscribe problem

Mon Nov 14 21:45:56 UTC 2005

What happened: I noticed that the builder completed a step, but failed to
move onto the next step - it just sat there doing nothing. So, I opened the
logs to around the time the last step finished, and I found these stack
traces.
 What appears to have happened is that the KeyError in the first case has
left the buildbot in an invalid state - it now has a status client
subscriber whose client slot is None, therefore it throws those later
exceptions.
 Any ideas what is happening here?
  2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] PB client detached
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Release)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Final)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Metrics)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Debug)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Documentation)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Gather Docs)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Lint)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Testing)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(Data Collection)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribing from
Builder(daily-stuff)
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribe from <
buildbot.status.builder.Status instance at 0x0122BC10>
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] unsubscribe from <
buildbot.status.builder.BuildStatus instance at 0x0364EB20>
2005/11/14 11:57 Pacific Standard Time [Broker,34,IP] Traceback (most recent
call last):
File "Z:\Tools\PythonLibs\Python24\twisted\internet\win32eventreactor.py",
line 196, in _runAction
self._disconnectSelectable(fd, closed, action == 'doRead')
File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py", line 232,
in _disconnectSelectable
selectable.readConnectionLost(f)
File "Z:\Tools\PythonLibs\Python24\twisted\internet\tcp.py", line 391, in
readConnectionLost
self.connectionLost(reason)
File "Z:\Tools\PythonLibs\Python24\twisted\internet\tcp.py", line 402, in
connectionLost
protocol.connectionLost(reason)
--- <exception caught here> ---
File "Z:\Tools\PythonLibs\Python24\twisted\spread\pb.py", line 641, in
connectionLost
notifier()
File "Z:\Tools\PythonLibs\buildbot\status\client.py", line 571, in <lambda>
lambda p=p,mind=mind: p.detached(mind))
File "Z:\Tools\PythonLibs\buildbot\status\client.py", line 376, in detached
s.unsubscribe(self)
File "Z:\Tools\PythonLibs\buildbot\status\builder.py", line 1064, in
unsubscribe
if self.updates[receiver] is not None:
exceptions.KeyError:
<buildbot.status.client.StatusClientPerspectiveinstance at 0x03948580>

2005/11/14 11:57 Pacific Standard Time [-] Traceback (most recent call
last):
File "Z:\Tools\PythonLibs\Python24\twisted\scripts\_twistw.py", line 44, in
runApp
app.runReactorWithLogging(config, oldstdout, oldstderr)
File "Z:\Tools\PythonLibs\Python24\twisted\application\app.py", line 128, in
runReactorWithLogging
reactor.run()
File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py", line 200,
in run
self.mainLoop()
File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py", line 208,
in mainLoop
self.runUntilCurrent()
--- <exception caught here> ---
File "Z:\Tools\PythonLibs\Python24\twisted\internet\base.py", line 533, in
runUntilCurrent
call.func(*call.args, **call.kw <http://call.kw>)
File "Z:\Tools\PythonLibs\buildbot\status\builder.py", line 1054, in
sendETAUpdate
receiver.buildETAUpdate(self, self.getETA())
File "Z:\Tools\PythonLibs\buildbot\status\client.py", line 474, in
buildETAUpdate
self.client.callRemote("buildETAUpdate",
exceptions.AttributeError: 'NoneType' object has no attribute 'callRemote'
 ( ... some time passes ... )

2005/11/14 12:01 Pacific Standard Time [-] Unhandled error in Deferred:
2005/11/14 12:01 Pacific Standard Time [-] Traceback (most recent call
last):
File "Z:\Tools\PythonLibs\buildbot\process\step.py", line 580, in finished
self.step_status.stepFinished(results)
File "Z:\Tools\PythonLibs\buildbot\status\builder.py", line 876, in
stepFinished
w.callback(self)
File "Z:\Tools\PythonLibs\Python24\twisted\internet\defer.py", line 254, in
callback
self._startRunCallbacks(result)
File "Z:\Tools\PythonLibs\Python24\twisted\internet\defer.py", line 315, in
_startRunCallbacks
self._runCallbacks()
--- <exception caught here> ---
File "Z:\Tools\PythonLibs\Python24\twisted\internet\defer.py", line 328, in
_runCallbacks
self.result = callback(self.result, *args, **kw)
File "Z:\Tools\PythonLibs\buildbot\status\builder.py", line 1146, in
_stepFinished
w.stepFinished(self, step, results)
File "Z:\Tools\PythonLibs\buildbot\status\client.py", line 490, in
stepFinished
self.client.callRemote("stepFinished",
exceptions.AttributeError: 'NoneType' object has no attribute 'callRemote'
 ( ... a short time later ... )
 2005/11/14 12:01 Pacific Standard Time [-] <SlaveLock(ps2_tool)[slave2]
23604200> is claimed
2005/11/14 12:01 Pacific Standard Time [-] Unhandled error in Deferred:
2005/11/14 12:01 Pacific Standard Time [-] Traceback (most recent call
last):
File "Z:\Tools\PythonLibs\Python24\twisted\internet\defer.py", line 117, in
maybeDeferred
result = f(*args, **kw)
File "Z:\Tools\PythonLibs\buildbot\process\step.py", line 484, in startStep
d.addCallback(self._startStep_2)
File "Z:\Tools\PythonLibs\Python24\twisted\internet\defer.py", line 216, in
addCallback
callbackKeywords=kw)
File "Z:\Tools\PythonLibs\Python24\twisted\internet\defer.py", line 207, in
addCallbacks
self._runCallbacks()
--- <exception caught here> ---
File "Z:\Tools\PythonLibs\Python24\twisted\internet\defer.py", line 328, in
_runCallbacks
self.result = callback(self.result, *args, **kw)
File "Z:\Tools\PythonLibs\buildbot\process\step.py", line 505, in
_startStep_2
self.step_status.stepStarted()
File "Z:\Tools\PythonLibs\buildbot\status\builder.py", line 822, in
stepStarted
self.build.stepStarted(self)
File "Z:\Tools\PythonLibs\buildbot\status\builder.py", line 1135, in
stepStarted
receiver = w.stepStarted(self, step)
File "Z:\Tools\PythonLibs\buildbot\status\client.py", line 481, in
stepStarted
self.client.callRemote("stepStarted",
exceptions.AttributeError: 'NoneType' object has no attribute 'callRemote'

( ... and so on, for each event, until I restart the buildbot ... )
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20051114/5905dfde/attachment.html>