[Buildbot-devel] Multi-distro Buildbot success story

[John Morris]

The configuration is pretty ugly, but it'll get cleaned up over time. 
Hope it didn't sprain your eyeballs.

Link added.  Thanks for looking!

	John


On 05/11/2014 09:56 PM, Dustin J. Mitchell wrote:
> This is a great write-up, and a nice use of Buildbot - thank you!
>
>  From a skim through the configuration, it looks like you've made great
> use of programmatic configuration, loading from JSON and building the
> configuration objects from there.
>
> Do you mind adding a link to http://trac.buildbot.net/wiki/SuccessStories ?
>
> Dustin
>
> On Wed, May 7, 2014 at 1:55 PM, John Morris <john at zultron.com> wrote:
>> The dust hasn't completely settled, but I'm excited to show off the
>> multi-distro, multi-arch Buildbot we use to test contributions to the
>> Machinekit open-source project (http://www.machinekit.io).  I thought
>> our instance might be interesting in this forum since it's publicly
>> viewable and is perhaps a fairly complex example.  See it here:
>>
>> http://buildbot.dovetail-automata.com/
>>
>> And the configuration (still a mess):
>>
>> https://github.com/zultron/machinekit-buildbot
>>
>> The Buildbot currently builds for four distros, Wheezy, EL6, and Fedora
>> 19+20; Jessie will be added soon.  It builds 32- and 64- bit x86
>> architectures, with ARM on the way.  The unit test build slaves also run
>> one of two real-time kernels, with a third to be added.
>>
>> The project is Machinekit, a machine controller based on LinuxCNC,
>> requiring a real-time thread environment.  We recently extended
>> Machinekit to support not only the RTAI RT kernel that LinuxCNC
>> originally supports, but also PREEMPT_RT, Xenomai userland and Xenomai
>> kernel threads, and non-RT POSIX threads, corresponding to the '*-tst'
>> builders on the Buildbot (RTAI currently absent).  The RTOS work
>> included revamping the build and run-time systems to build loadable
>> modules for all available RT environments in a single './configure &&
>> make', and run-time scripts detect the running RT environment and load
>> the appropriate thread modules.
>>
>> This so-called 'Universal Build' enables an interesting Buildbot
>> workflow.  The 'builder0' slave is configured with 8 CPUs, and runs the
>> build in a chroot environment for each of the eight distro+arch
>> combinations ('*-bld' builders), building modules for four threads
>> systems.  Each of these build results is then sent to two matching test
>> slaves, one running RT_PREEMPT, the other Xenomai kernels, which then
>> run unit tests to cover all thread systems.  The 'builder0' slave also
>> builds packages for each distro+arch ('*-pkg' builders), as well as a
>> large documentation tree for each distro (arch-independent '*-doc'
>> builders).
>>
>> This Buildbot is specially important to the Machinekit project, which
>> uses the C4 process (http://rfc.zeromq.org/spec:22) to minimize hurdles
>> of contribution submission and acceptance in hopes of scaling up
>> community development.  By C4, patches only need to follow a few simple,
>> formal rules to be accepted, including compiling and testing cleanly.
>> With the variety of platforms requiring testing, Buildbot saves time and
>> effort for developers.  Also because C4 standards don't require value
>> judgements by reviewers, unlike standards of many other projects,
>> comprehensive testing on all supported platforms is relatively more
>> critical.
>>
>> Please feel free to poke around the Buildbot web GUI and configuration.
>>    Suggestions/criticisms are very welcome!  (Please CC: me on list replies.)
>>
>> Thanks-
>>
>>          John
>>
>>
>> ------------------------------------------------------------------------------
>> Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
>> • 3 signs your SCM is hindering your productivity
>> • Requirements for releasing software faster
>> • Expert tips and advice for migrating your SCM now
>> http://p.sf.net/sfu/perforce
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel