[Buildbot-commits] [Buildbot] #2252: Reconfig after adding step results in buildException

[Buildbot]

#2252: Reconfig after adding step results in buildException
-------------------+---------------------
Reporter:  tmidd   |       Owner:
    Type:  defect  |      Status:  closed
Priority:  major   |   Milestone:  0.8.6
 Version:  0.8.6   |  Resolution:  fixed
Keywords:          |
-------------------+---------------------
Changes (by dustin):

 * milestone:  undecided => 0.8.6


Old description:

> I added an extra step to a builder and then reconfig'ed the master. The
> step I added was automatically named MasterShellCommand_2 which pushed
> the previous owner of that name to MasterShellCommand_3 (which didn't
> exist previously). There were no builds running on the builder when I did
> the reconfig.
>
> The next time the builder finished it threw the following exception and
> failed to "end" correctly - that is, subsequent builds are pending even
> though there is no active build. Restarting the master seems to have
> fixed the problem.
>
> 2012-03-22 14:22:25+1300 [-]  setting expectations for next time
> 2012-03-22 14:22:25+1300 [-] <Build balance-incr>.buildException
> 2012-03-22 14:22:25+1300 [-] Unhandled Error
>         Traceback (most recent call last):
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 396,
> in _stepDone
>             return self.startNextStep()
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 386,
> in startNextStep
>             d.addCallback(self._stepDone, s)
>           File "/usr/local/lib/python2.7/site-
> packages/Twisted-12.0.0-py2.7-linux-
> x86_64.egg/twisted/internet/defer.py", line 301, in addCallback
>             callbackKeywords=kw)
>           File "/usr/local/lib/python2.7/site-
> packages/Twisted-12.0.0-py2.7-linux-
> x86_64.egg/twisted/internet/defer.py", line 290, in addCallbacks
>             self._runCallbacks()
>         --- <exception caught here> ---
>           File "/usr/local/lib/python2.7/site-
> packages/Twisted-12.0.0-py2.7-linux-
> x86_64.egg/twisted/internet/defer.py", line 551, in _runCallbacks
>             current.result = callback(current.result, *args, **kw)
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 396,
> in _stepDone
>             return self.startNextStep()
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 383,
> in startNextStep
>             return self.allStepsDone()
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 486,
> in allStepsDone
>             return self.buildFinished(text, self.result)
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 517,
> in buildFinished
>             self.builder.setExpectations(self.progress)
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/process/builder.py", line 450,
> in setExpectations
>             self.expectations.update(progress)
>           File "/usr/local/lib/python2.7/site-
> packages/buildbot-0.8.6-py2.7.egg/buildbot/status/progress.py", line 295,
> in update
>             old = self.times[name]
>         exceptions.KeyError: 'MasterShellCommand_3'
>
> It looks like it's trying to find the new step in a dict that wasn't
> updated correctly by the reconfig, although I haven't had a proper look
> through the code yet to confirm exactly what's going wrong.

New description:

 I added an extra step to a builder and then reconfig'ed the master. The
 step I added was automatically named MasterShellCommand_2 which pushed the
 previous owner of that name to MasterShellCommand_3 (which didn't exist
 previously). There were no builds running on the builder when I did the
 reconfig.

 The next time the builder finished it threw the following exception and
 failed to "end" correctly - that is, subsequent builds are pending even
 though there is no active build. Restarting the master seems to have fixed
 the problem.

 {{{
 2012-03-22 14:22:25+1300 [-]  setting expectations for next time
 2012-03-22 14:22:25+1300 [-] <Build balance-incr>.buildException
 2012-03-22 14:22:25+1300 [-] Unhandled Error
         Traceback (most recent call last):
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 396, in
 _stepDone
             return self.startNextStep()
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 386, in
 startNextStep
             d.addCallback(self._stepDone, s)
           File "/usr/local/lib/python2.7/site-
 packages/Twisted-12.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
 line 301, in addCallback
             callbackKeywords=kw)
           File "/usr/local/lib/python2.7/site-
 packages/Twisted-12.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
 line 290, in addCallbacks
             self._runCallbacks()
         --- <exception caught here> ---
           File "/usr/local/lib/python2.7/site-
 packages/Twisted-12.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
 line 551, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 396, in
 _stepDone
             return self.startNextStep()
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 383, in
 startNextStep
             return self.allStepsDone()
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 486, in
 allStepsDone
             return self.buildFinished(text, self.result)
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/process/build.py", line 517, in
 buildFinished
             self.builder.setExpectations(self.progress)
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/process/builder.py", line 450,
 in setExpectations
             self.expectations.update(progress)
           File "/usr/local/lib/python2.7/site-
 packages/buildbot-0.8.6-py2.7.egg/buildbot/status/progress.py", line 295,
 in update
             old = self.times[name]
         exceptions.KeyError: 'MasterShellCommand_3'
 }}}

 It looks like it's trying to find the new step in a dict that wasn't
 updated correctly by the reconfig, although I haven't had a proper look
 through the code yet to confirm exactly what's going wrong.

--

-- 
Ticket URL: <http://trac.buildbot.net/ticket/2252#comment:3>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation