[Buildbot-devel] lazy logs a bit too lazy and que'ed requests not being processed

Tue Oct 14 08:54:17 UTC 2014

Yep it seems we have a slightly modified version of the code. In particular
the _start method of RemoteCommand:

def _start(self):
        if 'stdio' not in self.logs or 'stdio' not in self.delayedLogs:
            log.msg("RemoteCommand (%s) is running a command, but "
                    "it isn't being logged to anything. This seems unusual."
                    % self)
        self.updates = {}
        self._startTime = util.now()

        # This method only initiates the remote command.
        # We will receive remote_update messages as the command runs.
        # We will get a single remote_complete when it finishes.
        # We should fire self.deferred when the command is done.
        d = self.remote.callRemote("startCommand", self, self.commandID,
                                   self.remote_command, self.args)
        return d

with the stuff in red being the added part.

I really could not figure it out as we have that source checked in in our
version control. And the the first revision already contains that change.
I'll download 0.8.6 again and start with a fresh install.

Chers,
 Francesco

On Tue, Oct 14, 2014 at 2:05 AM, Dustin J. Mitchell <dustin at v.igoro.us>
wrote:

>
> https://github.com/buildbot/buildbot/blob/eight/master/buildbot/status/web/logs.py
> is where the streaming takes place.
>
> As for "is running a command, but it isn't being logged to anything.
> This seems unusual.", I don't see that message anywhere in the
> Buildbot codebase.  Presumably it's some custom code of yours?
>
> Dustin
>
> On Mon, Oct 13, 2014 at 12:56 PM, Francesco Di Mizio
> <francescodimizio at gmail.com> wrote:
> > Sorry to bother you again. I've tried to set up a new instance of
> buildbot
> > just for testing, which is being served without a proxy.
> >
> > Well logs streaming is not working yet... from the logs I gather:
> >
> > <RemoteShellCommand... is running a command, but it isn't being logged to
> > anything. This seems unusual.
> >
> > On Mon, Oct 13, 2014 at 11:50 AM, Francesco Di Mizio
> > <francescodimizio at gmail.com> wrote:
> >>
> >> Yep you are right I'll see their mailing list.
> >>
> >> One very last thing: Should I need to provide them with more details,
> >> could you please point me to the relevant code for streaming logs?
> >>
> >> FDM
> >>
> >> On Sat, Oct 11, 2014 at 5:36 AM, Dustin J. Mitchell <dustin at v.igoro.us>
> >> wrote:
> >>>
> >>> At this point it's an Apache question, so you might have better luck
> >>> seeking out Apache experts (of which I, at least, am not one).
> >>>
> >>> Dustin
> >>>
> >>> On Fri, Oct 10, 2014 at 6:57 AM, Francesco Di Mizio
> >>> <francescodimizio at gmail.com> wrote:
> >>> > No sure of this can be of any help:
> >>> >
> >>> > apache's error.log:
> >>> >
> >>> > [Fri Oct 10 12:54:17 2014] [debug] proxy_util.c(1818): proxy: grabbed
> >>> > scoreboard slot 1 in child 7231 for worker proxy:reverse
> >>> > [Fri Oct 10 12:54:17 2014] [debug] proxy_util.c(1837): proxy: worker
> >>> > proxy:reverse already initialized
> >>> > [Fri Oct 10 12:54:17 2014] [debug] proxy_util.c(1934): proxy:
> >>> > initialized
> >>> > single connection worker 1 in child 7231 for (*)
> >>> >
> >>> >
> >>> > On Thu, Oct 9, 2014 at 11:03 AM, Francesco Di Mizio
> >>> > <francescodimizio at gmail.com> wrote:
> >>> >>
> >>> >> I can't seem to find a solution for this, I'll give up for now.
> Really
> >>> >> ran
> >>> >> out of ideas.
> >>> >>
> >>> >> Initially in my email I mentioned one more issue other than the one
> >>> >> we've
> >>> >> been discussing.
> >>> >>
> >>> >> I'll just start a new thread not to mess this thread up.
> >>> >>
> >>> >> Cheers
> >>> >>
> >>> >> On Wed, Oct 8, 2014 at 4:25 PM, Francesco Di Mizio
> >>> >> <francescodimizio at gmail.com> wrote:
> >>> >>>
> >>> >>> Well according to the docs I don't see how dropping that to 512
> could
> >>> >>> have any effect as based on the docs:
> >>> >>>
> >>> >>> The size must be at least 8192.
> >>> >>>
> >>> >>> I gave it a try anyway with no visible results.
> >>> >>>
> >>> >>> Forget about lazy logs in the buildbot sense of the term, I am not
> >>> >>> using
> >>> >>> them either.
> >>> >>>
> >>> >>> Thanks for your help :)
> >>> >>>
> >>> >>> On Wed, Oct 8, 2014 at 3:55 PM, Charles Lepple <clepple at gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> On Oct 8, 2014, at 9:46 AM, Francesco Di Mizio
> >>> >>>> <francescodimizio at gmail.com> wrote:
> >>> >>>>
> >>> >>>> In a previous reply, Charles wrote:
> >>> >>>>
> >>> >>>> In Apache, I dropped the proxy buffer from the default of 8 kB to
> >>> >>>> 512
> >>> >>>> byte.
> >>> >>>>
> >>> >>>> I am assuming you are talking about the ProxyReceiveBufferSize.
> Am I
> >>> >>>> assuming right?
> >>> >>>>
> >>> >>>>
> >>> >>>> actually, it was ProxyIOBufferSize, which also defaults to 8kB:
> >>> >>>>
> >>> >>>>
> >>> >>>>
> http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxyiobuffersize
> >>> >>>>
> >>> >>>> I don't remember why I adjusted that one versus the receive
> buffer.
> >>> >>>> This
> >>> >>>> is with Apache 2.2.
> >>> >>>>
> >>> >>>> Also, I am not using "lazy logs" - I didn't realize that was a
> >>> >>>> specific
> >>> >>>> term. I was just trying to follow normal stdio logs with less
> delay.
> >>> >>>>
> >>> >>>> --
> >>> >>>> Charles Lepple
> >>> >>>> clepple at gmail
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>
> >>> >>
> >>> >
> >>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20141014/c8b80f21/attachment.html>