[Buildbot-devel] Multiple change sources and scheduler issues

[Dustin J. Mitchell]

On Wed, Jul 23, 2008 at 11:16 AM, Axel Hecht <l10n.moz at googlemail.com> wrote:
> We can name the changesources something like "l10n-af", "l10n-ar",
> "l10n-be", but doing
> dict(forrest="l10n", repo="af") would be prettier. So all our en-US
> scheduler would pay attention for dict(repo="mozilla-central"), while
> the l10n scheduler would pay attention to dict(forrest="l10n") and
> would then set build properties based on the "repo" color to trigger
> repacks for only a particular language.

Hmm, so this is even more complex than I thought.  I thought you were
basically doing an intersection of the keys for the colordict of a
Change and a Scheduler, and triggering the build if the intersection
was nonempty.  But it looks like you're treating the dictionary on a
Change as a set of attributes, and the dictionary on a Scheduler as a
smaller set of attributes which must match -- basically a logical AND.
 I'm not sure how you'd implement an OR -- I think by adding a new
attribute on the Changes.

I'm resistant to this idea because it differs from the other means of
connecting one bbot component to the next, which is based on lists of
names.  I'm big on consistency and minimal complexity.

We, too, have a large configuration (in our case with lots of various
types of builders), but our solution is to dynamically generate the
names in master_cfg.py.  For example, we have:

for arch in all_arches:
    c['builders'].append({
        'name': "archtest-%s" % arch,
        'slavenames': [ buildslave.botname
                for buildslave in buildslaves
                if buildslave.arch == arch ],
        'builddir': "archtest-%s" % arch,
        'category' : "archtest",
        'factory': archtest_factory,
          })

I think your situation could be solved similarly by creating compound
names for each ChangeSource (something like "%(forrest)-%(repo)") and
then dynamically creating the lists of names passed to the scheduler.

This suits your need in a complex situation, and will suit others'
complex situations without having to puzzle out how to map their
situation into your dict-partition model.  But it leaves the basic
functionality simple and easy to describe.

What do you think?

Dustin

-- 
Storage Software Engineer
http://www.zmanda.com