[Buildbot-devel] Blocker step not working as expected - what is wrong?

[Greg Ward]

On 21 March 2012, André Anjos said:
> I just implemented a blocker step (sub-classed as required) to one of
> our builds. This build has to wait for some other step in another
> builder.

What version of BuildBot are you using? I use Blocker successfully (no
surprise, since I wrote it) with 0.7.12. When Dustin merged it in
about a year ago, all its tests passed, but it requires manual testing
-- so it might have suffered bitrot and stopped working at some point.
;-(

> Here is the code for the Blocker derived class that worked for me:
> 
> class WaitDatabase(Blocker):
>   """Implements a waiting step that waits for the database to be installed"""
> 
>   name = "db_sync"
> 
>   def buildsMatch(self, buildStatus1, buildStatus2):
> 
>     n2 = buildStatus2.getBuilder().getName()
>     depbuilder = self.upstreamSteps[0][0]
>     depstep = self.upstreamSteps[0][1]
> 
>     if buildStatus2.getBuilder().getName() == depbuilder and \
>         (not buildStatus2.isFinished()):

I believe your first test (getName() == depbuilder) is a tautology:
IIRC, Blocker always passes the BuildStatus object for one of your
upstream steps as buildStatus2. Add some logging; I suspect you will
see that this condition is always true. If not, I'm probably
misremembering something.

So that means that you're really only testing

  if not buildStatus2.isFinished():

i.e. you assume that buildStatus1 and buildStatus2 match if #2 is
still running. That's a pretty weak test!

>       for step in buildStatus2.getSteps():
>         if step.getName() == depstep:
>           if not step.isFinished():
>             # we must wait
>             self._log("Waiting for unfinished %s step on %s", depstep,
> depbuilder)
>             return True

Hmmmm. Looks to me like a reinvention of what Blocker should already
be doing. So either your match condition is wrong, or Blocker is
broken.

For the record, here's how I use Blocker:

class MyBlocker(blocker.Blocker):
    """
    Custom version of Blocker that matches builds based on our
    ims_build_id property.
    """
    def buildsMatch(self, buildStatus1, buildStatus2):
        try:
            return (buildStatus1.getProperty("ims_build_id") ==
                    buildStatus2.getProperty("ims_build_id"))
        except KeyError:
            # One or both builds is missing that rather important build property.
            return False

ims_build_id is a derived from the primary key in a MySQL table that
records all of our builds. One row in that table identifies what
BuildBot sees as three builds (linux, windows, mac). I use Blocker to
ensure that the Windows builder does not try to build an installer
around our .jar file until the .jar file is compiled on Linux and
copied to Windows, and then later to ensure that the Linux builder
does not try to build an RPM around the Windows installer until *it*
has been built and copied to Linux. (Don't ask. It makes my head spin
even to contemplate our crazy build.)

The key is that ims_build_id is completely independent of BuildBot; it
is derived from nothing in the BuildBot universe. That's how I *know*
this Linux and Windows build go together.

        Greg
-- 
Greg Ward                                http://www.gerg.ca/
Pointers are Arrays; Code is Data; Time is Money