[Buildbot] #3081: buildbot.test.integration.test_try_client is flaky

Buildbot trac trac at buildbot.net
Thu Dec 25 15:18:59 UTC 2014


#3081: buildbot.test.integration.test_try_client is flaky
-------------------+-----------------------
Reporter:  Ben     |       Owner:  dustin
    Type:  defect  |      Status:  assigned
Priority:  major   |   Milestone:  0.9.0
 Version:  master  |  Resolution:
Keywords:          |
-------------------+-----------------------

Comment (by dustin):

 OK, got another failure overnight:
 {{{
 2014-12-25 04:46:19+0000 [-] -->
 buildbot.test.integration.test_master.RunMaster.test_master2 <--
 2014-12-25 04:46:19+0000 [-] Loading configuration from
 '/home/ubuntu/buildbot/_trial_temp/basdir/master.cfg'
 2014-12-25 04:46:19+0000 [-] Setting up database with URL
 'sqlite:/state.sqlite'
 2014-12-25 04:46:19+0000 [-] setting database journal mode to 'wal'
 2014-12-25 04:46:19+0000 [-] using SQLAlchemy-Migrate version 0.7.2
 2014-12-25 04:46:19+0000 [-] Initializing empty database
 2014-12-25 04:46:22+0000 [-] Starting BuildMaster -- buildbot.version:
 0.9.0
 2014-12-25 04:46:22+0000 [-] Loading configuration from
 '/home/ubuntu/buildbot/_trial_temp/basdir/master.cfg'
 2014-12-25 04:46:22+0000 [-] adding 1 new changesources, removing 0
 2014-12-25 04:46:22+0000 [-] adding 1 new builders, removing 0
 2014-12-25 04:46:22+0000 [-] trying to load status pickle from
 /home/ubuntu/buildbot/_trial_temp/basdir/testy/builder
 2014-12-25 04:46:22+0000 [-] no saved status pickle, creating a new one
 2014-12-25 04:46:22+0000 [-] added builder testy with tags []
 2014-12-25 04:46:22+0000 [-] adding scheduler 'all'
 2014-12-25 04:46:23+0000 [-] adding scheduler 'force'
 2014-12-25 04:46:23+0000 [-] No web server configured on this master
 2014-12-25 04:46:23+0000 [-] adding 1 new slaves, removing 0
 2014-12-25 04:46:23+0000 [-]   File
 "/home/ubuntu/buildbot/sandbox/bin/trial", line 18, in <module>
             run()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/scripts/trial.py", line 615, in run
             test_result = trialRunner.run(suite)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 729, in run
             return self._runWithoutDecoration(test,
 self._forceGarbageCollection)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 756, in _runWithoutDecoration
             run()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 751, in <lambda>
             run = lambda: suite.run(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 219, in run
             TestSuite.run(self, result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asyncrunner.py", line 36, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 178, in run
             super(LoggedSuite, self).run(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asyncrunner.py", line 36, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 151, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 151, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 151, in run
             test(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asynctest.py", line 119, in __call__
             return self.run(*args, **kwargs)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_synctest.py", line 943, in run
             _collectWarnings(self._warnings.append,
 self._runFixturesAndTest, result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_synctest.py", line 182, in _collectWarnings
             result = f(*args, **kwargs)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asynctest.py", line 300, in _runFixturesAndTest
             self._wait(d)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asynctest.py", line 378, in _wait
             reactor.run()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/base.py", line 1192, in run
             self.mainLoop()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/base.py", line 1201, in mainLoop
             self.runUntilCurrent()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/base.py", line 797, in runUntilCurrent
             f(*a, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1142, in _inlineCallbacks
             deferred.callback(e.value)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1142, in _inlineCallbacks
             deferred.callback(e.value)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1142, in _inlineCallbacks
             deferred.callback(e.value)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1102, in _inlineCallbacks
             deferred.callback(None)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1102, in _inlineCallbacks
             deferred.callback(None)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1102, in _inlineCallbacks
             deferred.callback(None)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
             result = g.send(result)
           File
 "/home/ubuntu/buildbot/master/buildbot/buildslave/manager.py", line 94, in
 reconfigServiceWithBuildbotConfig
             new_config)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1237, in unwindGenerator
             return _inlineCallbacks(None, gen, Deferred())
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
             result = g.send(result)
           File "/home/ubuntu/buildbot/master/buildbot/util/service.py",
 line 43, in reconfigServiceWithBuildbotConfig
             yield svc.reconfigServiceWithBuildbotConfig(new_config)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1237, in unwindGenerator
             return _inlineCallbacks(None, gen, Deferred())
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
             result = g.send(result)
           File "/home/ubuntu/buildbot/master/buildbot/buildslave/base.py",
 line 225, in reconfigServiceWithBuildbotConfig
             yield self.registration.update(new, new_config)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1237, in unwindGenerator
             return _inlineCallbacks(None, gen, Deferred())
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
             result = g.send(result)
           File
 "/home/ubuntu/buildbot/master/buildbot/buildslave/manager.py", line 53, in
 update
             global_config.protocols['pb']['port'])
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1237, in unwindGenerator
             return _inlineCallbacks(None, gen, Deferred())
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
             result = g.send(result)
           File
 "/home/ubuntu/buildbot/master/buildbot/buildslave/protocols/pb.py", line
 49, in updateRegistration
             portStr, username, password, self._getPerspective)
           File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
 58, in register
             disp = self.dispatchers[portstr] = Dispatcher(portstr)
           File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
 124, in __init__
             self.port = strports.listen(portstr, self.serverFactory)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/application/strports.py", line 99, in listen
             return getattr(reactor, 'listen'+name)(*args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/posixbase.py", line 495, in listenTCP
             p.startListening()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/tcp.py", line 993, in startListening
 log.msg(''.join(traceback.format_list(traceback.extract_stack())))

 2014-12-25 04:46:23+0000 [-] PBServerFactory starting on 40494
 2014-12-25 04:46:23+0000 [-] Starting factory
 <twisted.spread.pb.PBServerFactory instance at 0x7f75d848e8c0>
 2014-12-25 04:46:23+0000 [-] BuildMaster is running
 2014-12-25 04:46:23+0000 [-] (TCP Port 40494 Closed)
 2014-12-25 04:46:23+0000 [-] Stopping factory
 <twisted.spread.pb.PBServerFactory instance at 0x7f75d848e8c0>
 2014-12-25 04:46:23+0000 [-]   File
 "/home/ubuntu/buildbot/sandbox/bin/trial", line 18, in <module>
             run()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/scripts/trial.py", line 615, in run
             test_result = trialRunner.run(suite)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 729, in run
             return self._runWithoutDecoration(test,
 self._forceGarbageCollection)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 756, in _runWithoutDecoration
             run()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 751, in <lambda>
             run = lambda: suite.run(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 219, in run
             TestSuite.run(self, result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asyncrunner.py", line 36, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 178, in run
             super(LoggedSuite, self).run(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asyncrunner.py", line 36, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 151, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 151, in run
             test(result)
           File "/usr/lib/python2.7/unittest/suite.py", line 70, in
 __call__
             return self.run(*args, **kwds)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/runner.py", line 151, in run
             test(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asynctest.py", line 119, in __call__
             return self.run(*args, **kwargs)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_synctest.py", line 943, in run
             _collectWarnings(self._warnings.append,
 self._runFixturesAndTest, result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_synctest.py", line 182, in _collectWarnings
             result = f(*args, **kwargs)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asynctest.py", line 300, in _runFixturesAndTest
             self._wait(d)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/trial/_asynctest.py", line 378, in _wait
             reactor.run()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/base.py", line 1192, in run
             self.mainLoop()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/base.py", line 1201, in mainLoop
             self.runUntilCurrent()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/base.py", line 797, in runUntilCurrent
             f(*a, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 382, in callback
             self._startRunCallbacks(result)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
             self._runCallbacks()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
             result = g.send(result)
           File "/home/ubuntu/buildbot/master/buildbot/util/service.py",
 line 173, in _activityPoll
             yield self.activate()
           File "/home/ubuntu/buildbot/master/buildbot/changes/pb.py", line
 152, in activate
             self._register(port)
           File "/home/ubuntu/buildbot/master/buildbot/changes/pb.py", line
 164, in _register
             self.getPerspective)
           File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
 58, in register
             disp = self.dispatchers[portstr] = Dispatcher(portstr)
           File "/home/ubuntu/buildbot/master/buildbot/pbmanager.py", line
 124, in __init__
             self.port = strports.listen(portstr, self.serverFactory)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/application/strports.py", line 99, in listen
             return getattr(reactor, 'listen'+name)(*args, **kw)
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/posixbase.py", line 495, in listenTCP
             p.startListening()
           File "/home/ubuntu/buildbot/sandbox/local/lib/python2.7/site-
 packages/twisted/internet/tcp.py", line 993, in startListening
 log.msg(''.join(traceback.format_list(traceback.extract_stack())))

 2014-12-25 04:46:23+0000 [-] PBServerFactory starting on 57307
 2014-12-25 04:46:23+0000 [-] Starting factory
 <twisted.spread.pb.PBServerFactory instance at 0x7f75d8438440>
 2014-12-25 04:46:24+0000 [-] BuildMaster is stopped
 2014-12-25 04:46:24+0000 [-] Main loop terminated.
 }}}

--
Ticket URL: <http://trac.buildbot.net/ticket/3081#comment:9>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation


More information about the bugs mailing list