[Buildbot-devel] Re: sharing changesources

[Thomas Vander Stichele]

Hi,

> > From: Thomas Vander Stichele
> > 
> > I have three builds using the same maildir mailbox as a changesource.
> > It looks to me like this is a problem - not all of them get all the
> > notifications.  Is this the wrong way to do it, or should the
> > changesource code be fixed to handle this ?
> 
> Yeah, that's a problem. If you have multiple MaildirSource objects watching
> the same maildir, they will fight over each message: only one will win, and
> the winner will be random (determined either by when the message arrives
> relative to the polling intervals, or which dnotify handler the kernel
> chooses to fire first).

Sometimes it even triggered two builds.  I split it up with a .forward
file, creating a mailbox per master.


> The "right way" to do this is to have just one changesource per maildir, and
> let multiple builders share that single changesource. Now, stating it like
> that implies that there is an arbitrary N-to-M map from ChangeSource
> instances to Builders, which is not currently the case. The buildbot config
> file lets you define a set of ChangeSources, but each Change from those
> sources is broadcast to all Builders, which are then free to deal with the
> Change as they please (ignore, unimportant, or important).
> 
> If there were a mapping like this, it would move buildbot a little bit closer
> to being able to handle multiple projects in a single buildmaster instance.

Well, in my case, I just wanted to separate the three current branches
we have in GStreamer into separate buildbots.  So CVS mails just get
sent out to one address, which then locally .forward's to three
different folders now as per your suggestion.

> I'm not yet convinced this is a good goal, but I can see advantages to it
> (one would be sharing the status web server and the status irc bot, rather
> than having a dozen different URLs to deal with).

That could make sense, but in that case it'd be nice to have a condensed
HTML view.  The waterfall really only usefully shows you 8 builds at a
glance, and in the case of GStreamer+deps that's definately not enough.

Here are two additional problems I've run into:

a) the timestamp of the CVS commit is being used for the checkout when
the build gets triggered. most of the times, this timestamp seems to be
*right before* the change was commited; ie. cvs checks out a tree from
*right before* the change was accepted into the tree.  So when you fix
something, the build actually ends up failing since it checks out code
before the fix.  Did you ever see this happen ? Is there an easy way for
me to nudge the timestamp forward a second ?

b) for the GStreamer builds, we have a dependency stack where for
example gst-plugins depends on gstreamer.  I set up interlocks and that
works.  However, as part of the gstreamer build, before the update, a
"make uninstall" is done (to make sure that when stuff gets removed from
the build, the install doesn't end up crufted with leftovers) and a make
install at the end if the build succeeds.  Two things can go wrong here:
- a forced build (through the webinterface) of gstreamer ends up making
a triggered build of plugins fail.  Should the plugins build be
interlocked due to the running build of gstreamer ?
- a failed gstreamer core build will make a subsequent plugins-only
build fail as well since the gstreamer build ran make uninstall and due
to the error didn't run make install, causing configure to fail for the
plugins build.  Is there a way for the plugins build to know that the
last core build failed and thus it should not trigger ?

Thanks
Thomas

Dave/Dina : future TV today ! - http://www.davedina.org/
<-*- thomas (dot) apestaart (dot) org -*->
resistance is low when I'm feeling bored
what I thought was fun isn't fun anymore
<-*- thomas (at) apestaart (dot) org -*->
URGent, best radio on the net - 24/7 ! - http://urgent.fm/