[Buildbot-devel] Compressing build logs, input wanted

[Axel Hecht]

Hi all,

I pushed a few more changesets onto the branch. I'm now having a
c['logCompressionLimit'] configuration variable, which gives a minimal size
of log files to compress. The default is set to 4k, which seems to be the
block size on linux, and that seems a good lower bound, as we're really
bothered about disk real estate, right? Setting the variable to False
switches all log compression off, which I use for most tests.

I don't enable compression in most tests because the log compression piece
is now in a separate thread via deferToThread. I keep the file moving and
removing in a callback in the main thread to avoid concurrency issues here.
There is a chance to wait for the log compression by getting the new
Deferred I return from stepFinished(), but that's not propagated, so for
tests that don't explicitly do, I run into unclean reactor shutdowns.

I'm currently running my bug26 branch locally as a test run for my local
installs, and that works kinda nicely.

Axel

2009/1/20 Axel Hecht <l10n.moz at googlemail.com>

> Hi all,
>
> I just pushed a git branch, http://github.com/Pike/buildbot/commits/bug26,
> to compress logs post-facto with bz2.
>
> Right now, it does so rather unconditionally, with a hidden pref in
> LogFile.compressLogs. I guess there would be some performance hits on the
> master for large logs, see my comments in
> http://buildbot.net/trac/ticket/26#comment:16.
>
> I'd like to know what others would expect, and who's relying on what in the
> logfiles.
>
> Axel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20090127/15233b29/attachment.html>