[Buildbot-devel] Dependent schedulers not being triggered

[Brian Warner]

Kenneth Lareau <Ken.Lareau at nominum.com> writes:

> The problem was user misconception in two different areas.  The first
> was that the person who'd set up the dependent _scheduler_ (note it's
> not plural) thought he'd actually set up a set of schedulers for each
> product we support; that as not the case, as our schedulers are config-
> ured one for each slave, not for each builder.  The reason for this is
> our build system can't easily (or safely) do simultaneous builds (for
> reasons I won't get into, but the system's limitation here is far, far
> outweighed by its benefits).

As noted by Darragh, you could use Locks to insure that only one build is
running at a time, which might be more robust. A single buildslave will
cheerfully service multiple *Builders* at the same time (so you could have as
many simultaneous Builds running as there are Builders which use that slave);
SlaveLocks are the way to reduce this parallelism.. particularly important
for, say, benchmarking runs :).

> The second was a misconception that if builders in a given scheduler
> suceeded, related builders in the depen- dent scheduler would also succeed;
> the documentation very clearly does state this to NOT be the case.

Right, the downstream scheduler will fire (thus triggering any Builders that
are connected to it) if and only if every Build that was triggered by the
upstream scheduler has finished and did not result in a FAILURE status. The
code that implements this logic is in
buildbot.buildset.BuildSet.requestFinished, about line 59.

Basically the upstream scheduler creates a BuildSet that contains a number of
BuildRequests, each BuildRequest results in a Build, the BuildSet tells the
BuildRequests that it wants to hear about the Builds finishing (by
subscribing the requestFinished() method), and the BuildSet accumulates the
results, then eventually notifies *its* subscribers (people who called
BuildSet.waitUntilSuccess or .waitUntilFinished).

The downstream scheduler asks the upstream scheduler to tell it when
BuildSet.waitUntilFinished has fired by invoking the upstream's
subscribeToSuccessfulBuilds() method. This subscription causes the downstream
scheduler's upstreamBuilt() method to be invoked (with a SourceStamp) every
time the upstream scheduler sees a BuildSet complete with a result of
SUCCESS.


As I understand it, what you want is to have a single upstream scheduler
trigger builders A,B,C, and then have a downstream Dependent scheduler fire
when A and B have succeeded (and C has finished, ignoring whether C succeeded
or failed).

If I'm correct in that understanding, then I think the easiest (but not
necessarily the best) thing to do is to make yourself a new upstream
scheduler. Have it inherit from Scheduler(), but override the
buildSetFinished method. Instead of only looking at the overall results
(bss.getResults()), do something like this:

 from buildbot.status.builder import WARNINGS, SUCCESS
 builders_I_care_about = ["a", "b"]
 should_run_downstream = True
 for req in bss.getBuildRequests():
     req_status = req.status
     builder_name = req_status.getBuilderName()
     if builder_name not in builders_I_care_about:
         continue
     for build in req_status.getBuilds():
         if build.getResults() not in (WARNINGS, SUCCESS):
             should_run_downstream = False
 if should_run_downstream:
     for w in self.successWatchers:
         w(bss.getSourceStamp())

It's a bit weird, I'll admit. Basically you're looking inside the BuildSet to
get the BuildRequests, then inside those to find the Builds (really
BuildStatus instances), then asking those about their results, and only
firing the downstream scheduler (self.successWatchers) if they were all
non-failing. The weirdness comes from 1) the status APIs not being convenient
to giving you quite enough information, and 2) the fact that a single
BuildRequest might have multiple Builds (all on the same Builder) associated
with it (although I honestly can't remember why I made a provision for this
right now.. I'm sure it came up somewhere, maybe interrupted builds or
something).

The slightly more difficult (but somewhat cleaner) approach would be to allow
the downstream scheduler to specify which builders it cares about. To do
this, I'd enhance BaseUpstreamScheduler to have a self.finishedWatchers list
(and associated subscribeToFinishedBuilds() method), which would fire (*with
the BuildSet*) when the BuildSet had finished, regardless of whether the
BuildSet thinks of itself as succeeding or failing. Then I'd modify BuildSet
to let you get at the BuildStatus objects of the component Builds more
easily. Then I'd change the downstream Dependent scheduler to accept a list
of "builder names that I should care about", and have it apply the same logic
as the sample code I wrote above. That way, the upstream scheduler doesn't
need to be aware of the peculiar likes and dislikes of the downstream
scheduler at all.. that information can be completely contained in the
Dependent scheduler.


hope that helps.. let me know how what you end up doing.

cheers,
 -Brian