[Buildbot-devel] Compressing build logs, input wanted

[Jean-Paul Calderone]

On Tue, 20 Jan 2009 20:04:00 +0100, Marcus Lindblom <macke at yar.nu> wrote:
>Axel Hecht wrote:
>> Hi all,
>>
>> I just pushed a git branch,
>> http://github.com/Pike/buildbot/commits/bug26, to compress logs
>> post-facto with bz2.
>>
>> Right now, it does so rather unconditionally, with a hidden pref in
>> LogFile.compressLogs. I guess there would be some performance hits on
>> the master for large logs, see my comments in
>> http://buildbot.net/trac/ticket/26#comment:16.
>>
>> I'd like to know what others would expect, and who's relying on what in
>> the logfiles.
>
>We're just reading them as text, so no comment on that part.
>
>It seems to make sense to compress post-facto in a separate thread, then
>do an atomic rename (from log.bz2.part to log.bz2)
>
>However, wouldn't it make more sense to gzip them, so that they could be
>sent in compressed format directly to browsers that accepted gzip format?
>
>I'd also wish for pluggable compression (so that I could use 7z, which
>is fast and I like it :)

I don't know if this helps at all, but <http://twistedmatrix.com/trac/ticket/727>.

Jean-Paul