[users at bb.net] StopIteration Exceptions in LineConsumerLogObserver

[Yngve N. Pettersen]

Hi again,

I just did a small bit of PoC testing, and based on that testing I suspect  
that the problem is that LineConsumerLogObserver.finishReceived is called  
too early from LogObserver.gotData because data is None, before all data  
have actually been added to the log.

The next line add then triggers a StopIteration exception.

PoC:

================
def foo():
   while True:
     try:
       x = yield

       print("X: %d" % x)
     except StopIteration:
       print("Finished!")
       return

generator = foo()

next(generator)

feed = generator.send

feed(1)
feed(2)
feed(3)

generator.close()

feed(4)
================

feed(4) will fail with the exception StopIteration

On Wed, 13 Jan 2021 17:52:51 +0100, Yngve N. Pettersen <yngve at vivaldi.com>  
wrote:

> Hi,
>
> We are using a class based on steps.ShellCommandNewStyle to process log  
> output using LineConsumerLogObserver from a number of tests.
>
> After the v2.10 upgrade (from v2.7) I have noticed several cases of  
> Exceptions being thrown and being registered as such in the Build status  
> information. One such case, missing handling of GeneratorExit, has been  
> fixed.
>
> But today a lot of these test runs started failing (next on todo list is  
> finding out why), and was reported terminated due to an exception.
>
> The exception turns out to be due to a StopIteration from  
> LineConsumerLogObserver.headerLineReceived (at least in the case I  
> investigated). My guess is that the "problem line" is one of the two  
> last lines in the log, which reports the exit status of the task.
>
> The log handler function is written like the logConsumer function in  
> <http://docs.buildbot.net/current/developer/cls-logobserver.html#buildbot.process.logobserver.LineConsumerLogObserver>
>
> Based on what I've quickly skimmed about StopIteration, it should be  
> handled by the caller of the logConsumer function. This suggests that  
> LineConsumerLogObserver or its caller need to be updated.
>
>
> BTW, this is not new in v2.10, this exception also happened in v2.7  
> based on my grep of the log files. I guess the reason it is now showing  
> up as an issue is that these exceptions are reported upwards to the step.
>
>


-- 
Sincerely,
Yngve N. Pettersen
Vivaldi Technologies AS