[Buildbot-devel] Confused about dependent builds

[Greg Ward]

Hi all --

Either I'm confused about how dependent builds are supposed to work, or I'm
trying to drag buildbot in a direction it doesn't want to go.  I think
the nub of
my problem is that it's not *builds* that are dependent, but *schedulers*.
Let me try to explain.

The reason I think I need dependent builds is that part of our build
must run on
Linux, and part must run on Windows.  I only want to checkout the tree once, so
I want to do that on a Linux buildslave, copy the tree to a Windows
buildslave, and
then let the two builds run concurrently.  So I think I need four builders:

1) on Linux: checkout tree, generate some code, and tar it up
2) on Windows: download and unpack the tarball, run Windows build
3) on Linux: run Linux build
4) on Linux: download products of Windows build and do final packaging

The dependency graph is obvious (in "make" syntax):

  2: 1
  3: 1
  4: 2 3

I think that only build 1 needs to be controlled by "normal" Schedulers --
the others should only run when their dependencies are satisfied.  So that
leads me to something like

  sched1 = Scheduler("sched1", ...)
  sched2 = Dependent("sched2", sched1, ...)
  sched3 = Dependent("sched3", sched1, ...)
  sched4 = Dependent("sched4", ?sched2 or sched3?, ...)

Uh-oh #1: how do I make build 4 depend on the outcome of more than one
other build?

Uh-oh #2: what happens I want multiple ways to trigger a build?  E.g.
it doesn't matter
if this is a production build triggered by manual action or a nightly
build triggered by
the clock: build 2 and 3 should not start until build 1 is done, and
build 4 should not start
until 2 and 3 are done.  But since it's schedulers that depend on
other schedulers,
rather than builds depending on other builds, this leads to an
explosion of dependencies:

  sched1_manual = ManuallyTriggeredScheduler("sched1_manual", ...)
  sched2_manual = Dependent("sched2_manual", sched1_manual, ...)
  sched3_manual = Dependent("sched3_manual", sched1_manual, ...)
  sched4_manual = Dependent("sched4_manual", ?sched2_manual or
sched3_manual?, ...)

  sched1_nightly = Nightly("sched1_nightly", ...")
  sched2_nightly = Dependent("sched2_nightly", sched1_nightly, ...)
  sched3_nightly = Dependent("sched3_nightly", sched1_nightly, ...)
  sched4_nightly = Dependent("sched4_nightly", ?sched2_nightly or
sched3_nightly?, ...)

Blech!  That's obviously not gonna fly.  I suppose I coud paper over
it by factoring out
a little function to generate the dependent schedulers for each "main"
scheduler, but
still.  And I still don't know how to express that build 4 must not
start until 2 and 3
complete successfully.

Any clues?

Greg