[Buildbot-devel] Compressing build logs, input wanted

[Marcus Lindblom]

Axel Hecht wrote:
> Hi all,
> 
> I just pushed a git branch, 
> http://github.com/Pike/buildbot/commits/bug26, to compress logs 
> post-facto with bz2.
> 
> Right now, it does so rather unconditionally, with a hidden pref in 
> LogFile.compressLogs. I guess there would be some performance hits on 
> the master for large logs, see my comments in 
> http://buildbot.net/trac/ticket/26#comment:16.
> 
> I'd like to know what others would expect, and who's relying on what in 
> the logfiles.

We're just reading them as text, so no comment on that part.

It seems to make sense to compress post-facto in a separate thread, then 
do an atomic rename (from log.bz2.part to log.bz2)

However, wouldn't it make more sense to gzip them, so that they could be 
sent in compressed format directly to browsers that accepted gzip format?

I'd also wish for pluggable compression (so that I could use 7z, which 
is fast and I like it :)

Cheers,
/Marcus