[Buildbot-devel] Buildbot can do tests

[Tim Flink]

On Thu, 18 Jun 2015 09:58:45 +0000
Pierre Tardy <tardyp at gmail.com> wrote:

> Hi,
> 
> I see a lot of misunderstanding about buildbot being a tool only
> designed for doing builds and cannot run a testfarm. (
> https://plus.google.com/+ThomasPetazzoni/posts/YUo9ohgboSJ)
> 
> Indeed, buildbot is initially created to do builds, and all the
> glossary is around builds. We got buildsets that turn into
> buildrequests that turn into builds, and set buildproperties.
> 
> Actually, buildbot is really a generic job scheduler, and is great at
> managing slaves, and storing the logs of arbitrary jobs.
> 
> Some of my colleagues are working on building a TestFarm manager using
> buildbot nine.
> This is running great, and they are having very nice results, thanks
> to the plugin design of the UI.
> But one of the feedback of their users is that it is written 'build'
> everywhere in the UI, while they are only controlling testfarm, using
> builds that have been done in another buildbot master.

We're in a similar boat - we make heavy use of buildbot for what we're
calling task automation (think test automation but far more generic -
along the lines of job automation) for Fedora. Our system [1] has been
in production [2] for about 9 months now - the load is somewhat low
right now but I expect both the load and the complexity of the system to
increase significantly after our next major release.

[1] http://fedoraproject.org/wiki/Taskotron
[2] https://taskotron.fedoraproject.org/taskmaster/waterfall

I've been meaning to write up a more detailed explanation of how we're
using buildbot and how it's becoming more critical for our ongoing
automated test efforts in Fedora but until then, figured I could at
least chime in to this thread :)

We've really made the "build" generic - it will clone a git repository
based on how the build itself is triggered. After that clone, the task
contained in the repo is executed and results are submitted to a
central location. In the 99000+ "builds" that have been done in our
production buildbot instance so far, I don't think that a single one
would be considered a build in the traditional sense.

This is a bit impromptu but thanks for making such a flexible
scheduling and execution framework available. It's helping us to
increase the number of automated checks run on Fedora during
development and enabling us to focus on the bits that are testing
Fedora instead of spending all of our time writing a framework for
scheduling and executing those tests/tasks.

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://buildbot.net/pipermail/devel/attachments/20150619/8ee352c0/attachment.bin>