[Buildbot-devel] Storing compile/test results in a database?

[Marcus Lindblom]

Kristian Nielsen wrote:
> I know that it has already been discussed to use an SQL database for backend
> storage of Buildbot state and history, and that the idea is for this to go in
> 1.0. However, I have a need to get some data in a database now, and my needs
> are also somewhat different, so I wanted to bounce off some ideas and hear if
> there is already some existing effort that I could build on.
> 
> We are using Buildbot for automatic testing of MariaDB, a comunity branch of
> MySQL. The main thing here is to run the test suite. There are on the order of
> 1000 individual test cases run over a number of different branches. Just
> maintaining the test suite is a major effort, and we need a tool to help with
> this.
> 
> Basically I want to store every test case failure in a database along with
> information about branch, revision, error output, etc. This will allow to
> easily create additional webpages, a facility to search for previous failures
> across different builds, etc. (At MySQL AB I wrote an internal tool for this,
> and the database was > 1 GB in size).
> 
> My idea is basically to put in custom LogObserver hooks and maybe subclass
> build steps as necessary to get access to the information, just adding code to
> do the appropriate SQL INSERT statements. The actual code for this will be
> fairly specific to the way the MySQL test suite works.
> 
> But I would like to hear if someone else already did something like this? Or
> if there already exists some generic support for this in Buildbot, or interest
> in using such generic support if I were to implement it?

There was a django/bb integration posted on the list a few months back. 
IIRC the author simply ran something like a LogObserver to populate 
Django models (i.e. into a database). Thus letting buildbot handle it's 
own data but outputting to Django to allow a separate system to do more 
than what buildbot's basic web services offer.

There was no deferreds or non-blocking stuff invovled. Wether due to 
lack of need or time, I'm not sure.

cheers,
/Marcus