[Buildbot-devel] big custom logs horking the status web server?

[Stephen Davis]

On Aug 8, 2005, at 1:48 PM, Brian Warner wrote:

>> This has been working fine for a while but I just enabled every
>> warning I could and now the webserver status page is choking when I
>> click on the "warnings" log.  The error page generated by the
>> webserver is at the end of this email and it's pretty long.
>
> Ah, got it. When you use addCompleteLog(), you get a single huge  
> chunk in the
> file that gets saved to disk. This file is saved as a series of  
> netstrings,
> and Twisted's standard netstring-parsing class happens to have a  
> built-in
> limit on how large of a string it will accept. The default is 100k,  
> so any
> one-chunk log that is bigger than this will blow up when you try to  
> view it.
> The error message is confused by the fact that I'm using  
> NetstringReceiver in
> a slightly weird way (the protocol responds to the "malformed"  
> oversized
> netstring by trying to hang up, but I didn't give it a .transport  
> attribute
> to do that with).
>
> The quick solution is to bump up this limitation:
>
> --- orig/buildbot/status/builder.py
> +++ mod/buildbot/status/builder.py
> @@ -45,6 +45,7 @@
>  ChunkTypes = ["stdout", "stderr", "header"]
>
>  class LogFileScanner(basic.NetstringReceiver):
> +    MAX_LENGTH=10*1000*1000
>      def __init__(self, chunk_cb, channels=[]):
>          self.chunk_cb = chunk_cb
>          self.channels = channels

Thanks, that worked.  Now I just need to fix all the warnings...

stephen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20050808/4b7a8711/attachment.html>