[Buildbot-commits] [Buildbot] #2019: graceful shutdown and triggered builds don't play as expected (0.8.4p1)

[Buildbot trac]

#2019: graceful shutdown and triggered builds don't play as expected (0.8.4p1)
--------------------+-----------------------
Reporter:  dberger  |       Owner:
    Type:  defect   |      Status:  reopened
Priority:  major    |   Milestone:  0.8.+
 Version:  0.8.4p1  |  Resolution:
Keywords:           |
--------------------+-----------------------

Comment (by buck):

 In short, if a graceful-shutdown command is sent to a buildbot, which then
 runs a trigger-and-wait step, that buildbot cluster will then cease to do
 anything useful indefinitely.

 The deadlock consists of three blocking parts:

  1. The graceful-shutdown command blocks, waiting for all builders to
 finish.
  2. The trigger-and-wait step blocks, waiting for its triggered builds to
 finish.
  3. The triggered build blocks, waiting for the graceful-shutdown to
 complete before *starting* itself.


 My essential strategy is to edit part 3. We'll make sure that builds which
 have steps blocking on them are allowed to run, even if we're in the
 process of a graceful shutdown.

 This image demonstrates the necessary behavior. When the "clean shutdown"
 command comes in, queued builds and non-blocking triggered builds pend
 until after the master restarts, while blocking triggered builds are
 allowed to run.

 [[Image(Selection_029.png)]]

 This image was generated on buildbot 0.8.3 using this patch: http://paste
 .pound-python.org/show/N3QKBAISKJpsLJWAWISD/

 I plan to provide a similar patch (in spirit if not implementation) and
 integration tests for buildbot/master branch (0.8.8).

-- 
Ticket URL: <http://trac.buildbot.net/ticket/2019#comment:9>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation