[Buildbot] #3466: nextSlave idiosyncrasies

[Buildbot trac]

#3466: nextSlave idiosyncrasies
------------------------+-----------------------
Reporter:  gtmacdonald  |      Owner:
    Type:  undecided    |     Status:  new
Priority:  major        |  Milestone:  undecided
 Version:  0.9.0b7      |   Keywords:
------------------------+-----------------------
 I was attempting to use nextSlave to assign incoming jobs to a specific
 slave. I had a situation where multiple builders of the same name were
 being started at once but I only wanted one per slave. I was able to get
 something working but because of a few issues I decided to setup an
 explicit 1-to-1 instead of dynamically assigning the slaves. There were a
 couple problems; one was that a recently assigned slave would appear in
 subsequent calls but not be identifiable as active yet, the other problem
 was that not all the possible slaves were being passed to nextSlave. I
 used a timer to get around the first problem. The second is out of my
 control so I switch to a 1-to-1 mapping which produces a lot of extra
 builders.

 It seems to me that nextSlave needs to wait for the slavebuilder state to
 update before subsequent calls and it needs to ensure that all possible
 slaves are included in the slaveBuilders. Also, if no slaves are selected
 by nextSlave it might be a good idea to try back later; it could be the
 user wants to stagger the builds or wait for properties to change. As it
 is now, if no builds are selected then that request is abandoned.

 Here's the code I was working with.

 {{{
 recentlyAssignedPerBatch = {}
 def getNextSlaveForBatch(builder, slaveBuilders, buildRequest):
     try:
         now = datetime.now()

         for slaveBuilder in slaveBuilders:
             activeSlaveBuilders = [sb for sb in
 slaveBuilder.slave.slavebuilders.values() if sb.isBusy()]
             for x in activeSlaveBuilders:
                 if builder.name in x.builder_name:
                     log.msg("***** skipping: " + x.builder_name + 'for: '
 + slaveBuilder.slave.name)
             if any([builder.name in x.builder_name for x in
 activeSlaveBuilders]):
                 continue

             recentlyAssigned =
 recentlyAssignedPerBatch.setdefault(builder.name, [])
             recentlyAssigned = [x for x in recentlyAssigned if (now -
 x[1]) < timedelta(minutes=5)]
             recentlyAssignedPerBatch[builder.name] = recentlyAssigned

             if slaveBuilder.slave.name in [x[0] for x in
 recentlyAssigned]:
                 log.msg('***** skipping %s because of recent assignment to
 builder %s.' % (slaveBuilder.slave.name, builder.name))
                 continue

             log.msg('***** assigning gp build to: ' +
 slaveBuilder.slave.name)
             recentlyAssigned.append((slaveBuilder.slave.name, now))

             return slaveBuilder
         log.msg('***** assigning gp build to: none')

         return None
     except Exception, e:
         log.msg('***** ' + str(e))
 }}}

--
Ticket URL: <http://trac.buildbot.net/ticket/3466>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation