[Buildbot] #3081: buildbot.test.integration.test_try_client is flaky
Buildbot trac
trac at buildbot.net
Thu Dec 25 15:18:59 UTC 2014
#3081: buildbot.test.integration.test_try_client is flaky
-------------------+-----------------------
Reporter: Ben | Owner: dustin
Type: defect | Status: assigned
Priority: major | Milestone: 0.9.0
Version: master | Resolution:
Keywords: |
-------------------+-----------------------
Comment (by dustin):
OK, got another failure overnight:
{{{
2014-12-25 04:46:19+0000 [-] -->
buildbot.test.integration.test_master.RunMaster.test_master2 <--
2014-12-25 04:46:19+0000 [-] Loading configuration from
'/home/ubuntu/buildbot/_trial_temp/basdir/master.cfg'
2014-12-25 04:46:19+0000 [-] Setting up database with URL
'sqlite:/state.sqlite'
2014-12-25 04:46:19+0000 [-] setting database journal mode to 'wal'
2014-12-25 04:46:19+0000 [-] using SQLAlchemy-Migrate version 0.7.2
2014-12-25 04:46:19+0000 [-] Initializing empty database
2014-12-25 04:46:22+0000 [-] Starting BuildMaster -- buildbot.version:
0.9.0
2014-12-25 04:46:22+0000 [-] Loading configuration from
'/home/ubuntu/buildbot/_trial_temp/basdir/master.cfg'
2014-12-25 04:46:22+0000 [-] adding 1 new changesources, removing 0
2014-12-25 04:46:22+0000 [-] adding 1 new builders, removing 0
2014-12-25 04:46:22+0000 [-] trying to load status pickle from
/home/ubuntu/buildbot/_trial_temp/basdir/testy/builder
2014-12-25 04:46:22+0000 [-] no saved status pickle, creating a new one
2014-12-25 04:46:22+0000 [-] added builder testy with tags []
2014-12-25 04:46:22+0000 [-] adding scheduler 'all'
2014-12-25 04:46:23+0000 [-] adding scheduler 'force'
2014-12-25 04:46:23+0000 [-] No web server configured on this master
2014-12-25 04:46:23+0000 [-] adding 1 new slaves, removing 0
2014-12-25 04:46:23+0000 [-] File
"/home/ubuntu/buildbot/sandbox/bin/trial", line 18, in <module>
run()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/scripts/trial.py", line 615, in run
test_result = trialRunner.run(suite)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 729, in run
return self._runWithoutDecoration(test,
self._forceGarbageCollection)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 756, in _runWithoutDecoration
run()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 751, in <lambda>
run = lambda: suite.run(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 219, in run
TestSuite.run(self, result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asyncrunner.py", line 36, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 178, in run
super(LoggedSuite, self).run(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asyncrunner.py", line 36, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 151, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 151, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 151, in run
test(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asynctest.py", line 119, in __call__
return self.run(*args, **kwargs)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_synctest.py", line 943, in run
_collectWarnings(self._warnings.append,
self._runFixturesAndTest, result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_synctest.py", line 182, in _collectWarnings
result = f(*args, **kwargs)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asynctest.py", line 300, in _runFixturesAndTest
self._wait(d)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asynctest.py", line 378, in _wait
reactor.run()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/base.py", line 1192, in run
self.mainLoop()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/base.py", line 1201, in mainLoop
self.runUntilCurrent()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/base.py", line 797, in runUntilCurrent
f(*a, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1142, in _inlineCallbacks
deferred.callback(e.value)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1142, in _inlineCallbacks
deferred.callback(e.value)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1142, in _inlineCallbacks
deferred.callback(e.value)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1102, in _inlineCallbacks
deferred.callback(None)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1102, in _inlineCallbacks
deferred.callback(None)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1102, in _inlineCallbacks
deferred.callback(None)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
result = g.send(result)
File
"/home/ubuntu/buildbot/master/buildbot/buildslave/manager.py", line 94, in
reconfigServiceWithBuildbotConfig
new_config)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1237, in unwindGenerator
return _inlineCallbacks(None, gen, Deferred())
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
result = g.send(result)
File "/home/ubuntu/buildbot/master/buildbot/util/service.py",
line 43, in reconfigServiceWithBuildbotConfig
yield svc.reconfigServiceWithBuildbotConfig(new_config)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1237, in unwindGenerator
return _inlineCallbacks(None, gen, Deferred())
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
result = g.send(result)
File "/home/ubuntu/buildbot/master/buildbot/buildslave/base.py",
line 225, in reconfigServiceWithBuildbotConfig
yield self.registration.update(new, new_config)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1237, in unwindGenerator
return _inlineCallbacks(None, gen, Deferred())
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
result = g.send(result)
File
"/home/ubuntu/buildbot/master/buildbot/buildslave/manager.py", line 53, in
update
global_config.protocols['pb']['port'])
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1237, in unwindGenerator
return _inlineCallbacks(None, gen, Deferred())
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
result = g.send(result)
File
"/home/ubuntu/buildbot/master/buildbot/buildslave/protocols/pb.py", line
49, in updateRegistration
portStr, username, password, self._getPerspective)
File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
58, in register
disp = self.dispatchers[portstr] = Dispatcher(portstr)
File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
124, in __init__
self.port = strports.listen(portstr, self.serverFactory)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/application/strports.py", line 99, in listen
return getattr(reactor, 'listen'+name)(*args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/posixbase.py", line 495, in listenTCP
p.startListening()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/tcp.py", line 993, in startListening
log.msg(''.join(traceback.format_list(traceback.extract_stack())))
2014-12-25 04:46:23+0000 [-] PBServerFactory starting on 40494
2014-12-25 04:46:23+0000 [-] Starting factory
<twisted.spread.pb.PBServerFactory instance at 0x7f75d848e8c0>
2014-12-25 04:46:23+0000 [-] BuildMaster is running
2014-12-25 04:46:23+0000 [-] (TCP Port 40494 Closed)
2014-12-25 04:46:23+0000 [-] Stopping factory
<twisted.spread.pb.PBServerFactory instance at 0x7f75d848e8c0>
2014-12-25 04:46:23+0000 [-] File
"/home/ubuntu/buildbot/sandbox/bin/trial", line 18, in <module>
run()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/scripts/trial.py", line 615, in run
test_result = trialRunner.run(suite)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 729, in run
return self._runWithoutDecoration(test,
self._forceGarbageCollection)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 756, in _runWithoutDecoration
run()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 751, in <lambda>
run = lambda: suite.run(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 219, in run
TestSuite.run(self, result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asyncrunner.py", line 36, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 178, in run
super(LoggedSuite, self).run(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asyncrunner.py", line 36, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 151, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 151, in run
test(result)
File "/usr/lib/python2.7/unittest/suite.py", line 70, in
__call__
return self.run(*args, **kwds)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/runner.py", line 151, in run
test(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asynctest.py", line 119, in __call__
return self.run(*args, **kwargs)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_synctest.py", line 943, in run
_collectWarnings(self._warnings.append,
self._runFixturesAndTest, result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_synctest.py", line 182, in _collectWarnings
result = f(*args, **kwargs)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asynctest.py", line 300, in _runFixturesAndTest
self._wait(d)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/trial/_asynctest.py", line 378, in _wait
reactor.run()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/base.py", line 1192, in run
self.mainLoop()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/base.py", line 1201, in mainLoop
self.runUntilCurrent()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/base.py", line 797, in runUntilCurrent
f(*a, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 382, in callback
self._startRunCallbacks(result)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
self._runCallbacks()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1155, in gotResult
_inlineCallbacks(r, g, deferred)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
result = g.send(result)
File "/home/ubuntu/buildbot/master/buildbot/util/service.py",
line 173, in _activityPoll
yield self.activate()
File "/home/ubuntu/buildbot/master/buildbot/changes/pb.py", line
152, in activate
self._register(port)
File "/home/ubuntu/buildbot/master/buildbot/changes/pb.py", line
164, in _register
self.getPerspective)
File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
58, in register
disp = self.dispatchers[portstr] = Dispatcher(portstr)
File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
124, in __init__
self.port = strports.listen(portstr, self.serverFactory)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/application/strports.py", line 99, in listen
return getattr(reactor, 'listen'+name)(*args, **kw)
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/posixbase.py", line 495, in listenTCP
p.startListening()
File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
packages/twisted/internet/tcp.py", line 993, in startListening
log.msg(''.join(traceback.format_list(traceback.extract_stack())))
2014-12-25 04:46:23+0000 [-] PBServerFactory starting on 57307
2014-12-25 04:46:23+0000 [-] Starting factory
<twisted.spread.pb.PBServerFactory instance at 0x7f75d8438440>
2014-12-25 04:46:24+0000 [-] BuildMaster is stopped
2014-12-25 04:46:24+0000 [-] Main loop terminated.
}}}
--
Ticket URL: <http://trac.buildbot.net/ticket/3081#comment:9>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation
More information about the bugs
mailing list