[Buildbot] #2895: Builds take a long time to get started on latent buildslaves

[Buildbot trac]

#2895: Builds take a long time to get started on latent buildslaves
------------------------+------------------------
Reporter:  agateau      |       Owner:
    Type:  enhancement  |      Status:  new
Priority:  major        |   Milestone:  undecided
 Version:  0.8.6p1      |  Resolution:
Keywords:  ec2          |
------------------------+------------------------

Comment (by dustin):

 Wow, four minutes to acquire !DeferredLocks?  I can run that in 118ms
 {{{
 #!python
 from twisted.internet import defer, reactor

 dl = defer.DeferredLock()

 @defer.inlineCallbacks
 def run():
     for i in range(8*109):
         yield dl.run(lambda: defer.succeed(None))
     reactor.stop()

 reactor.callWhenRunning(run)
 reactor.run()
 }}}

 Also, note that once a builder is in the list of pending builders (which
 is accessed without the lock), there's no locking operation to handle.

 I think the slow bit is sorting the builders, which calls
 `bldr.getOldestRequestTime`.  That occurs each time
 `resetPendingBuildersList` runs.

 Rather than just not checking for available builds when a latent slave
 starts up, which will leave latent slaves idle if there are already build
 requests pending, I think the better solution would be to use the fact
 that `maybeStartBuildsOn` can take an iterator of builder names, and add a
 `maybeStartBuildsOnBuilders(builders)` method to `BotMaster`.  Then the
 logic that ends up calling `maybeStartBuildsForBuilder` could be changed
 around to call `maybeStartBuildsForBuilders` with the whole list of
 builders.

 The first call to this method will acquire the lock and add all of those
 builders.  Subsequent calls (assuming checks of the builders are still
 pending) will do nothing.

--
Ticket URL: <http://trac.buildbot.net/ticket/2895#comment:4>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation