[Buildbot-devel] Blocker step not working as expected - what is wrong?

[André Anjos]

Thanks for the info, Greg.

> What version of BuildBot are you using? I use Blocker successfully (no
> surprise, since I wrote it) with 0.7.12. When Dustin merged it in
> about a year ago, all its tests passed, but it requires manual testing
> -- so it might have suffered bitrot and stopped working at some point.
> ;-(

BB version: 0.8.6 (first tried with 0.8.5, same problem there)

> I believe your first test (getName() == depbuilder) is a tautology:
> IIRC, Blocker always passes the BuildStatus object for one of your
> upstream steps as buildStatus2. Add some logging; I suspect you will
> see that this condition is always true. If not, I'm probably
> misremembering something.

Well, I did add some logging to get to that formula ;-) It took some
pocking on the code to understand what the Blocker did w/o a proper
example somewhere. The code can be read online here:

http://buildbot.net/buildbot/docs/current/reference/buildbot.steps.blocker-pysrc.html

The start method (begins at line 200) is doing all the magic. In short:

1) Using (what seems to be) the last builds in every builder
registered, check which the current build should go for (this happens
inside _getBuildStatus).

At line 142: Get list of probed builds (that what goes in buildStatus2)

all_builds = (builderStatus.buildCache.values() +
builderStatus.getCurrentBuilds())

2) Do the same for the steps on that build. If no build found, outcome
depends on what the user configured for the blocking step.

3) If something is identified, block on that with a subscription.

So, there seems to be a gap between what you are describing (what
seems pretty logical to me) and what we have implemented there. Am I
wrong? (I actually would prefer to be since I would like to use this
functionality ;-)

My problem is much simpler than yours. We have multiple builds for the
same software. It so happens that one of these builds is special as it
creates "stuff" that *may* be consumed by the other builds. If the
"special" build is running, I would like the other builds to wait
until new "stuff" is ready. Otherwise, the builds can just go on
happily (no blocking).

Andre

>
> So that means that you're really only testing
>
>  if not buildStatus2.isFinished():
>
> i.e. you assume that buildStatus1 and buildStatus2 match if #2 is
> still running. That's a pretty weak test!
>
>>       for step in buildStatus2.getSteps():
>>         if step.getName() == depstep:
>>           if not step.isFinished():
>>             # we must wait
>>             self._log("Waiting for unfinished %s step on %s", depstep,
>> depbuilder)
>>             return True
>
> Hmmmm. Looks to me like a reinvention of what Blocker should already
> be doing. So either your match condition is wrong, or Blocker is
> broken.
>
> For the record, here's how I use Blocker:
>
> class MyBlocker(blocker.Blocker):
>    """
>    Custom version of Blocker that matches builds based on our
>    ims_build_id property.
>    """
>    def buildsMatch(self, buildStatus1, buildStatus2):
>        try:
>            return (buildStatus1.getProperty("ims_build_id") ==
>                    buildStatus2.getProperty("ims_build_id"))
>        except KeyError:
>            # One or both builds is missing that rather important build property.
>            return False
>
> ims_build_id is a derived from the primary key in a MySQL table that
> records all of our builds. One row in that table identifies what
> BuildBot sees as three builds (linux, windows, mac). I use Blocker to
> ensure that the Windows builder does not try to build an installer
> around our .jar file until the .jar file is compiled on Linux and
> copied to Windows, and then later to ensure that the Linux builder
> does not try to build an RPM around the Windows installer until *it*
> has been built and copied to Linux. (Don't ask. It makes my head spin
> even to contemplate our crazy build.)
>
> The key is that ims_build_id is completely independent of BuildBot; it
> is derived from nothing in the BuildBot universe. That's how I *know*
> this Linux and Windows build go together.
>
>        Greg
> --
> Greg Ward                                http://www.gerg.ca/
> Pointers are Arrays; Code is Data; Time is Money
>



-- 
Dr. André Anjos
Idiap Research Institute
Centre du Parc - rue Marconi 19
CH-1920 Martigny, Suisse
Phone: +41 27 721 7763
Fax: +41 27 721 7712
http://andreanjos.org