[users at bb.net] Prioritizing try builders with prioritizeBuilders?

[Dan Kegel]

Here's my monkey-see, monkey-do solution after getting past infant
runtime errors.
In a very quick test, it did seem to run try builds before
already-pending non-try builds.
(It relies on the try builders being named a certain way; presumably
there's a better way to look at a builder and decide whether it's a
try builder.)

--- a/common/SimpleConfig.py
+++ b/common/SimpleConfig.py
@@ -2,6 +2,8 @@ from buildbot.plugins import *
 import buildbot.revlinks
 import buildbot.status.html

+from twisted.internet import defer
+from twisted.python.failure import Failure
 from twisted.python import log

 from BBDependencies import BBDependencyGraph
@@ -287,6 +289,19 @@ def
buildRequestsCompatibleAndAreTrickleDownRebuilds(builder, req1, req2):
         return True
     return False

+ at defer.inlineCallbacks
+def prioritizeBuilders(buildmaster, builders):
+    """Sort the given list of builders with buildbot's default sort,
+    then move try builders to the beginning of the list so
+    developers get crisp responses to their try build requests."""
+    try:
+        builders = yield defer.maybeDeferred(lambda:
+            buildmaster.botmaster.brd._defaultSorter(buildmaster, builders))
+    except Exception:
+        log.msg("Exception prioritizing builders; presorting failed")
+        log.err(Failure())
+    defer.returnValue(sorted(builders, key=lambda b: "trybuilder-"
not in b.name))
+
 class SimpleConfig(dict):
     """A buildbot master with a web status page and a 'force build' button,
     which reads public configuration from 'master.json'
@@ -396,6 +411,9 @@ class SimpleConfig(dict):
         # (e.g. during transitive trickle-down builds)
         self['mergeRequests'] =
buildRequestsCompatibleAndAreTrickleDownRebuilds

+        # Sort builders so try builders get first dibs on cpu.
+        self['prioritizeBuilders'] = prioritizeBuilders
+
         ####### PORT NUMBERS
         # It's hard to keep port numbers straight for multiple projects,
         # so let's assign each project a slot number,

On Sat, Mar 12, 2016 at 4:49 PM, Dan Kegel <dank at kegel.com> wrote:
> Hi!
> Our users would like try builders to have high priority.
> This looks like the way to do it:
> http://docs.buildbot.net/latest/manual/cfg-global.html#prioritizing-builders
> http://docs.buildbot.net/latest/manual/customization.html#builder-priority-functions
>
> Evidently using it is not trivial, though.
> Mozilla uses it, and had some trouble:
> https://bugzilla.mozilla.org/show_bug.cgi?id=984923
> https://github.com/mozilla/build-buildbot-configs/blob/master/mozilla/master_common.py
> Chromium used it, and had some trouble:
> https://bugs.chromium.org/p/chromium/issues/detail?id=397087
>
> Mozilla's prioritizeBuilders looked really complicated, but Chromium's
> was relatively simple:
>
>   from twisted.internet import defer
>   from twisted.python import log
>   from twisted.python.failure import Failure
>
> @defer.deferredGenerator
> def prioritizeBuilders(buildmaster, builders):
>       # Presort with default sorting function of buildbot. The default sorting
>       # takes the age of the latest build requests into account.
>       sorter = (lambda:
>           buildmaster.botmaster.brd._defaultSorter(buildmaster, builders))
>       wfd = defer.waitForDeferred(defer.maybeDeferred(sorter))
>       yield wfd
>       builders = wfd.getResult()
>     except:
>       log.msg("Exception prioritizing builders; presorting failed")
>       log.err(Failure())
>     yield sorted(builders, key=lambda b: builder_priorities.get(b.name, 0))
>
> where their own sort is added in the very last line; presumably in my
> case I'd make a function that returned 0 for normal builds and 1
> for try builds (or vice versa) and rely on sorted() to be stable.
>
> Is that the recommended approach?  Seems like this ought to be covered
> in the doc somehow.
> - Dan