[Buildbot-devel] Parse the stdio file for errors and warnings and add them to the waterfall page and on an email formatter

[Alexander O'Donovan-Jones]

Have you looked into the WarningCountingShellCommand class? It does pretty much what you're after.
________________________________________
From: Fulvio Cervone [fcervone at me.com]
Sent: 07 February 2011 21:16
To: buildbot-devel list
Subject: [Buildbot-devel] Parse the stdio file for errors and warnings and add them to the waterfall page and on an email formatter

I am getting kinda swamped in this.....

I've checked the manual and the mailing list, and seems that there is no quick way to do a grep on the stdio and just get the output and show it on the waterfall page, so from what i have seen, you have to use logfiles and LogObservers.

What i want to achieve is to get the errors and warnings from the build process and put them in 2 files on the waterfall page, and in case of errors to get also the output of the file into my customized email formatter.

I've found this example here on a post in the mailing list history:

=== 2

class BetterCompile(Compile):
command = ["make", "all"]

def createSummary(self, log):
# 'log' is the stdio LogFile, there might be others
text = StringIO(log.getText())
warnings = [line for line in text.readlines()
if line.startswith("warning:")]
self.addCompleteLog('warnings', "".join(warnings))

f.addStep(BetterCompile)

This would work for what i need (just have to do another parsing for the errors and i am done basically), but I am not clear how this works...

1) the class is created inheriting from Compile, I do not use compile but ShellCommand; may I do the same using ShellCommand? (changing of course the command)

2) when the def createSummary is done, self refers to the actual step (and the class of course), but log is coming from where? I was not able to find the declaration or assignment statement for logs.....it should come from the step itself right? In this case what kind of declaration should i do to get the object "log"?

I have done this so far to get the errors:

class BetterCompile(ShellCommand):
command = ["xcodebuild", "-configuration", "Debug"]

def createSummary(self, log):
# 'log' is the stdio LogFile, there might be others
text = StringIO(log.getText())
warnings = [line for line in text.readlines()
if line.startswith("warning:")]
self.addCompleteLog('warnings', "".join(warnings))

f.addStep(BetterCompile)

and when i do the checkconfig i don't get any error, but when i run the step it gives me an exception

2011-02-07 13:04:39-0800 [-] BuildStep.failed, traceback follows
2011-02-07 13:04:39-0800 [-] Unhandled Error
Traceback (most recent call last):
 File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 186, in addCallbacks
   self._runCallbacks()
 File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 328, in _runCallbacks
   self.result = callback(self.result, *args, **kw)
 File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 243, in callback
   self._startRunCallbacks(result)
 File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 312, in _startRunCallbacks
   self._runCallbacks()
--- <exception caught here> ---
 File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 328, in _runCallbacks
   self.result = callback(self.result, *args, **kw)
 File "/Library/Python/2.6/site-packages/buildbot-0.8.3p1-py2.6.egg/buildbot/process/buildstep.py", line 1093, in <lambda>
   d.addCallback(lambda res: self.createSummary(cmd.logs['stdio']))
 File "/masterbot/master.cfg", line 140, in createSummary
   text = StringIO(log.getText())
exceptions.NameError: global name 'StringIO' is not defined


Am I missing something or this example is not working? Is there any other way to just get the stdio file output and grep it for errors and warning, and then put the output in an object that can be accessed by the mail notifier and the waterfall page?

Thanks!