[Buildbot-devel] Parse the stdio file for errors and warnings and add them to the waterfall page and on an email formatter

[Philippe McLean]

The visual studio compile steps in buildbot/steps/vstudio.py also have an
example of doing this.

On Mon, Feb 7, 2011 at 1:55 PM, Alexander O'Donovan-Jones <
Alexander at ccpgames.com> wrote:

> Have you looked into the WarningCountingShellCommand class? It does pretty
> much what you're after.
> ________________________________________
> From: Fulvio Cervone [fcervone at me.com]
> Sent: 07 February 2011 21:16
> To: buildbot-devel list
> Subject: [Buildbot-devel] Parse the stdio file for errors and warnings and
> add them to the waterfall page and on an email formatter
>
> I am getting kinda swamped in this.....
>
> I've checked the manual and the mailing list, and seems that there is no
> quick way to do a grep on the stdio and just get the output and show it on
> the waterfall page, so from what i have seen, you have to use logfiles and
> LogObservers.
>
> What i want to achieve is to get the errors and warnings from the build
> process and put them in 2 files on the waterfall page, and in case of errors
> to get also the output of the file into my customized email formatter.
>
> I've found this example here on a post in the mailing list history:
>
> === 2
>
> class BetterCompile(Compile):
> command = ["make", "all"]
>
> def createSummary(self, log):
> # 'log' is the stdio LogFile, there might be others
> text = StringIO(log.getText())
> warnings = [line for line in text.readlines()
> if line.startswith("warning:")]
> self.addCompleteLog('warnings', "".join(warnings))
>
> f.addStep(BetterCompile)
>
> This would work for what i need (just have to do another parsing for the
> errors and i am done basically), but I am not clear how this works...
>
> 1) the class is created inheriting from Compile, I do not use compile but
> ShellCommand; may I do the same using ShellCommand? (changing of course the
> command)
>
> 2) when the def createSummary is done, self refers to the actual step (and
> the class of course), but log is coming from where? I was not able to find
> the declaration or assignment statement for logs.....it should come from the
> step itself right? In this case what kind of declaration should i do to get
> the object "log"?
>
> I have done this so far to get the errors:
>
> class BetterCompile(ShellCommand):
> command = ["xcodebuild", "-configuration", "Debug"]
>
> def createSummary(self, log):
> # 'log' is the stdio LogFile, there might be others
> text = StringIO(log.getText())
> warnings = [line for line in text.readlines()
> if line.startswith("warning:")]
> self.addCompleteLog('warnings', "".join(warnings))
>
> f.addStep(BetterCompile)
>
> and when i do the checkconfig i don't get any error, but when i run the
> step it gives me an exception
>
> 2011-02-07 13:04:39-0800 [-] BuildStep.failed, traceback follows
> 2011-02-07 13:04:39-0800 [-] Unhandled Error
> Traceback (most recent call last):
>  File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
> line 186, in addCallbacks
>   self._runCallbacks()
>  File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
> line 328, in _runCallbacks
>   self.result = callback(self.result, *args, **kw)
>  File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
> line 243, in callback
>   self._startRunCallbacks(result)
>  File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
> line 312, in _startRunCallbacks
>   self._runCallbacks()
> --- <exception caught here> ---
>  File
> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
> line 328, in _runCallbacks
>   self.result = callback(self.result, *args, **kw)
>  File
> "/Library/Python/2.6/site-packages/buildbot-0.8.3p1-py2.6.egg/buildbot/process/buildstep.py",
> line 1093, in <lambda>
>   d.addCallback(lambda res: self.createSummary(cmd.logs['stdio']))
>  File "/masterbot/master.cfg", line 140, in createSummary
>   text = StringIO(log.getText())
> exceptions.NameError: global name 'StringIO' is not defined
>
>
> Am I missing something or this example is not working? Is there any other
> way to just get the stdio file output and grep it for errors and warning,
> and then put the output in an object that can be accessed by the mail
> notifier and the waterfall page?
>
> Thanks!
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110207/f564d012/attachment.html>