[Buildbot-devel] RFC: using buildbot to manage crowd-sourced testing.

[Stefan Seefeld]

Hello,

it has been a while since I actively followed buildbot development, so
please forgive me if this question has been answered already. Consider
the following use-case:

Many Free Software projects rely on contributors not only to develop
code, but also to build and test it. Often, the software needs to be
tested on a wide range of platforms. This is challenging for traditional
CI environments, especially if the software is complex and building and
testing it takes up lots of resources.

So, instead of centralizing the CI jobs, one would crowd-source them.
Contributors would provide free cycles to a central CI scheduler that
offloads builds to those machines, and collects back (test) results.

Is something like this already possible with buildbot ? I can see a
couple of challenges, for example to clearly encapsulating the build
environment on the slave, to a) protect the contributor's hardware
against viruses, and b) allow the master to assert that the platform is
valid (according to a project-specific metric).

Ideally, projects would then set up a buildmaster whose only task is to
collect results, and provide self-contained buildslaves for all the
different platforms / configurations that are to be tested. Contributors
would then download a fully self-contained buildslave, install it on the
platform they'd like to contribute, define some resource limits they'd
like to contribute, and simply "start" the service.

I figure such a setup would be immensely useful for quite a number of
complex projects, such as boost (hi David !), which still struggle with
adequate build and test coverage.

Any thoughts ?

Thanks,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...