[Buildbot-devel] Links to Mozilla's code

[Justin Wood (Callek)]

Dustin J. Mitchell wrote:
> A few folks asked for links to Mozilla's Buildbot configs.  We're an
> open-source company, so it's all available.. but finding the needles
> in the haystack can be tricky.
> 
>   http://hg.mozilla.org/build/buildbot
> This is Mozilla's fork of Buildbot-0.8.2.  The changes are fairly minimal.
> 
>   http://hg.mozilla.org/build/buildbotcustom
> This collects custom implementations of steps, schedulers, factories, etc.
> 
>   http://hg.mozilla.org/build/buildbot-configs
> The Buildbot configuration itself.  How all of this comes together is
> a bit over my head.
> 
> I hope that helps someone out.  We're at the sprints now, and will be
> for the duration, if you have more questions!

Hello,

(for those who don't know me, I am "Callek"/Justin Wood from Mozilla Releng)

So to elaborate on this all for everyone here....

What dustin said is accurate though we also use a pre version of
buildbot for most of our slaves, which in practice doesn't change much
since every master is still on 0.8.2 (that 0.8.4-pre is in a branch of
that buildbot repo)

We also setup buildbot in a python virtualenv, with an added
site-package pointing at our tools repo helpers:

http://hg.mozilla.org/build/tools/file/default/lib/python

----

Now for the fit-together sense...

We have a _LOT_ of machines, so no single master does all our work.

There are two primary directories we use from buildbot-configs (this
repo is also shared with projects like SeaMonkey):
* http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla
* http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests

There are 3 basic types of masters we use:
* Schedulers
* Build
* Test

Scheduler masters are responsible for reading the changesources and
identifying what builds/tests should be scheduled. We drive those with
things like
http://hg.mozilla.org/build/buildbot-configs/file/a880563582c9/mozilla/scheduler_master.cfg

We can "setup" a master quite easily with our Makefile.setup, which
calls out to a script which knows what to symlink (e.g.
production_config.py to config.py)

All our masters are enumerated in
http://hg.mozilla.org/build/tools/file/358636da8294/buildfarm/maintenance/production-masters.json
which the buildbot-configs reads to know if it should omit certain
platforms/etc. (so we don't bloat the config of a master with slaves
that will never connect)

We then take files like
http://hg.mozilla.org/build/buildbot-configs/file/a880563582c9/mozilla/config.py
and build deeply nested var configs that help us dictate what
factories/settings should be present for our vast array of build
types/trees.

the bulk of the work happens in a file like
http://hg.mozilla.org/build/buildbot-configs/file/a880563582c9/mozilla/builder_master.cfg#l58
which calls into our buildbotcustom file "misc.py" calling things like
generateBranchObjects which takes a config for a particular branch
(tree) and then creates all the objects that are meant to be used on
"this master"

Whether that be a windows master, linux master, etc.

There are some known issues in this setup we wish to improve upon, but
this is a setup that at least is working for us right now, at the scale
we possess.

--
I hope this summary is both readable/undstandable. But I'm happy to help
clarify any Q's. Either here or in IRC (I'm frequently on
irc.mozilla.org/#releng but sometimes on freenode/#buildbot)

-- 
~Justin Wood (Callek)