[Buildbot-devel] buildbot dependent scheduler firing twice for each upstream build

Fri Dec 12 17:37:03 UTC 2008

Thus far, I've made the following change to my local install of
buildbot-0.7.9 to prevent the duplicate Dependent builds.

I know this isn't a real "fix" because the code I've commented out
here also exists in buildbot-0.7.7, a version that worked fine.
What the patch does is prevents the duplication of the successWatchers
list onto the new scheduler instance created by the AnyBranchScheduler.
This in turn prevents one extra call of my Dependent scheduler.

If anyone has any other suggestions, I'd be happy to hear them.
This has been difficult since I don't really know Python very well.

mark

--- buildbot-0.7.9/buildbot/scheduler.py        2008-09-16 09:02:31.000000000 -0700
+++ buildbot-0.7.9.new/buildbot/scheduler.py    2008-12-12 09:32:01.000000000 -0800
@@ -290,7 +290,10 @@ class AnyBranchScheduler(BaseUpstreamSch
                                       self.treeStableTimer,
                                       self.builderNames,
                                       self.fileIsImportant)
-            s.successWatchers = self.successWatchers
+            # Copying successWatchers into branch scheduler seems
+            #  to fire off an extra dependent build.
+            #
+            # s.successWatchers = self.successWatchers
             s.setServiceParent(self)
             # TODO: does this result in schedulers that stack up forever?
             # When I make the persistify-pass, think about this some more.





> 
> I've added some debug statements to the buildbot code in order to
> make some progress on this.
> 
> I first added a log message in BaseUpstreamScheduler::submitBuildSet
> 
>           log.msg ("%s.submitBuildSet (%s)" % (self.name, bs.reason))
> 
> to see if this function was indeed being called twice for my Dependent
> schedulers. Indeed, from the logs I saw this was true:
> 
>  2008/12/09 11:54 -0700 [-] announce4.submitBuildSet (None)
> 
>  2008/12/09 11:54 -0700 [-] announce4.submitBuildSet (None)
> 
> 
> I then added another log message in the buildSetFinished routine, here:
> 
>         if bss.getResults() == builder.SUCCESS:
>             ss = bss.getSourceStamp()
>             log.msg ("%s.buildSetFinished: calling %d success watchers" %
>                      (self.name, len (self.successWatchers)))
>             for w in self.successWatchers:
>                 w(ss)
> 
> to see if buildSetFinished was being called more than once, or if the
> annouce4 'watcher' was somehow added to the successWatchers list twice.
> I'm seeing the following after a build has finished:
> 
>  2008/12/09 11:54 -0700 [-] rpms.chaos-dchroot.buildSetFinished: calling 1 su
> ccess watchers
> 
>  2008/12/09 11:54 -0700 [-] rpms.buildSetFinished: calling 1 success watchers
> 
> 
> Somehow buildSetFinished is being called twice. The base name of this upstrea
> m
> Scheduler should be just 'rpms'. However, I am seeing an extra Scheduler 
> instance calling its buildSetFinished method, and that one has 'chaos-dchroot
> '
> appended to its name. In this case 'chaos-dchroot' should be the 'branch' 
> attribute of the buildSet.
> 
> Looking further back in the logs, I do see that submitBuildSet was called
> for each of these Schedulers:
> 
>  2008/12/09 11:53 -0700 [-] rpms.chaos-dchroot.submitBuildSet (None)
>  2008/12/09 11:53 -0700 [-] rpms.submitBuildSet (None)
> 
> Remember that the 'rpms' Scheduler is an AnyBranch scheduler, and from the 
> comments I see
> 
>     """This Scheduler will handle changes on a variety of branches. It will
>     accumulate Changes for each branch separately. It works by creating a
>     separate Scheduler for each new branch it sees."""
> 
> So it seems to me like the rpms.<branch> scheduler is only one that should be
>  
> active during these builds, if I understand that comment correctly.
> 
> So, does anyone know why this extra scheduler is active and triggers
> my dependent builds? BTW, I am _not_ seeing an extra actual build, just
> an extra trigger of the dependent build. 
> 
> mark
> 
> > Hi,
> > 
> > We've been using buildbot for automated builds for awhile now, and
> > I've recently upgraded from 0.7.7 to 0.7.9 in order to hopefully
> > get support for exclusive/shared (r/w) locks. 
> > 
> > However, once I upgraded the buildbot code -- using the same master.cfg --
> > I am running into a new problem. The dependent schedulers I use seem
> > to be firing two builds per one upstream build, instead of one.
> > 
> > The key portion of my master.cfg looks something like:
> > 
> > build = AnyBranchScheduler ("rpms", None, 0,
> >                             fileIsImportant=isImportant,
> >                             builderNames=['x86_64', 'i386'])
> > 
> > announce = Dependent ("announce", build, ['annnounce'])
> > 
> > 
> > What we had working in buildbot-0.7.7 was that a 'build' 
> > triggered builds on both x86-64 and i386 builders, and if
> > each of these completed successfully, the announcement
> > build was triggered.
> > 
> > However, now we see two announce builds triggered for each 'build'.
> > 
> > Have there been any changes in dependent schedulers of which we should
> > be aware, or any other places in master.cfg to check for errors on our
> > part? I tried looking in the twistd.log, but there are no clues as to
> > how the announce builder is being triggered.
> > 
> > Thanks,
> > mark
> > 
> > 
> > 
> > ---------------------------------------------------------------------------
> --
> > -
> > SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
> > The future of the web can't happen without you.  Join us at MIX09 to help
> > pave the way to the Next Web now. Learn more and register at
> > http://  ad.doubleclick.net/clk;208669438;13503038;i?http://  2009.visitmix
> .com
> > /
> > _______________________________________________
> > Buildbot-devel mailing list
> > Buildbot-devel at lists.sourceforge.net
> > https://  lists.sourceforge.net/lists/listinfo/buildbot-devel
> > 
> 
> 
> -----------------------------------------------------------------------------
> -
> SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
> The future of the web can't happen without you.  Join us at MIX09 to help
> pave the way to the Next Web now. Learn more and register at
> http:// ad.doubleclick.net/clk;208669438;13503038;i?http:// 2009.visitmix.com
> /
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https:// lists.sourceforge.net/lists/listinfo/buildbot-devel
> 



