[Buildbot-devel] Complex unit testing build step advice

[Sergey A. Lipnevich]

Hi All,

I am looking to implement a single build step for a unit testing routine
that produces pass/fail code, variable number of XML files with test
results, and a single binary file of code coverage data. I want the
coverage percentage to be in step's text in waterfall, and pass/fail
status to be reflected by color. These two requirements are easy I
think, I just have to override a couple of methods in a standard
builder. The difficult part is how to present the detailed test results
and detailed coverage data using thew new WebStatus (I'm on 0.7.6).
First of all, I don't want to generate HTML and keep it on disk, I'd
rather "decorate" step output on the fly, similarly to how TextLog class
does it by wrapping text in HTML tags. Then, number of test results
files is variable and is not known in advance, so I'm trying to think of
the best way to present them. E.g. should each be shown separately or
should they somehow be merged. If I needed to merge them for HTML
presentation only, not on disk, how can I do this in BuildBot? These are
XML as I said so there are some XSLTs I could reuse to show pretty bar
charts of test completion, but I'm not sure how to apply XSL
transformation on the fly in a "chunked" way so that Twisted is happy.
Finally, the code coverage database as in a proprietary format so it
requires a 3rd party tool to generate HTML. Again, I'd rather not keep
the resulting HTML file around, hence another question: how should this
be integrated into BuildBot's WebStatus?
Of course, the whole idea of not keeping intermediate HTML on disk may
backfire as the status may be too slow, so if you tried that or think
it's a bad idea please let me know! Details are not accessed very often
so I'm trying to strike a balance between saving disk space and
providing rich status view to those few who care :-).
Any ideas on how to do all of this better and in a way that's
BuildBot-friendly are very much appreciated.
Thank you!

Sergey.