[Buildbot-devel] sharing changesources

[Brian Warner]

> From: Thomas Vander Stichele
> 
> I have three builds using the same maildir mailbox as a changesource.
> It looks to me like this is a problem - not all of them get all the
> notifications.  Is this the wrong way to do it, or should the
> changesource code be fixed to handle this ?

Yeah, that's a problem. If you have multiple MaildirSource objects watching
the same maildir, they will fight over each message: only one will win, and
the winner will be random (determined either by when the message arrives
relative to the polling intervals, or which dnotify handler the kernel
chooses to fire first).

The "right way" to do this is to have just one changesource per maildir, and
let multiple builders share that single changesource. Now, stating it like
that implies that there is an arbitrary N-to-M map from ChangeSource
instances to Builders, which is not currently the case. The buildbot config
file lets you define a set of ChangeSources, but each Change from those
sources is broadcast to all Builders, which are then free to deal with the
Change as they please (ignore, unimportant, or important).

If there were a mapping like this, it would move buildbot a little bit closer
to being able to handle multiple projects in a single buildmaster instance.
I'm not yet convinced this is a good goal, but I can see advantages to it
(one would be sharing the status web server and the status irc bot, rather
than having a dozen different URLs to deal with). If we go this way, it
should still be easy to have a ChangeSource be broadcast to all builders
(rather than requiring an explicit map or subscription notation in the config
file), because one-project-one-buildmaster (but multiple builders, one per
architecture) is likely to remain a common case.

Such a change would mean modifying the ChangeSource parent class to have a
list of subscribed Builders.. at the moment everything gets funneled through
the ChangeMaster which then broadcasts to all Builders.


Now, if you mean that you've got multiple buildmasters, each with a
MaildirSource pointed at the same maildir, then it's the same problem without
an easy solution. Only one MaildirSource will get the message, the others
(whether in the same process or in another) will lose. If you want to deliver
those messages to multiple places, you'll want a front-end .qmail or .forward
that will replicate the message and deliver it into multiple distinct
maildirs, one per MaildirSource instance.


hope that helps,
 -Brian