[Buildbot-devel] Parse the stdio file for errors and warnings and add them to the waterfall page and on an email formatter

[Fulvio Cervone]

Thanks Alexander and Philippe

I've found (not without a lot of perils) the class called WarningCountingShellCommand;

http://buildbot.net/buildbot/docs/0.8.3/reference/buildbot.steps.shell.WarningCountingShellCommand-class.html

Once found the class, I tried to apply it instead of the ShellCommand, and actually I can get the warnings and errors (just have to modify the warning pattern)

Now if i wanna get 2 logs(warning and errors) should i run 2 steps or is there a way to specify more than one log, with different name? (as now the Waterfall page shows "warnings (n)" independently from what i use as search pattern)

BTW I use Xcode, is the vstudio.py still useful to get the gist of how this works?




On Feb 7, 2011, at 3:01 PM, Philippe McLean wrote:

> The visual studio compile steps in buildbot/steps/vstudio.py also have an example of doing this. 
> 
> On Mon, Feb 7, 2011 at 1:55 PM, Alexander O'Donovan-Jones <Alexander at ccpgames.com> wrote:
> Have you looked into the WarningCountingShellCommand class? It does pretty much what you're after.
> ________________________________________
> From: Fulvio Cervone [fcervone at me.com]
> Sent: 07 February 2011 21:16
> To: buildbot-devel list
> Subject: [Buildbot-devel] Parse the stdio file for errors and warnings and add them to the waterfall page and on an email formatter
> 
> I am getting kinda swamped in this.....
> 
> I've checked the manual and the mailing list, and seems that there is no quick way to do a grep on the stdio and just get the output and show it on the waterfall page, so from what i have seen, you have to use logfiles and LogObservers.
> 
> What i want to achieve is to get the errors and warnings from the build process and put them in 2 files on the waterfall page, and in case of errors to get also the output of the file into my customized email formatter.
> 
> I've found this example here on a post in the mailing list history:
> 
> === 2
> 
> class BetterCompile(Compile):
> command = ["make", "all"]
> 
> def createSummary(self, log):
> # 'log' is the stdio LogFile, there might be others
> text = StringIO(log.getText())
> warnings = [line for line in text.readlines()
> if line.startswith("warning:")]
> self.addCompleteLog('warnings', "".join(warnings))
> 
> f.addStep(BetterCompile)
> 
> This would work for what i need (just have to do another parsing for the errors and i am done basically), but I am not clear how this works...
> 
> 1) the class is created inheriting from Compile, I do not use compile but ShellCommand; may I do the same using ShellCommand? (changing of course the command)
> 
> 2) when the def createSummary is done, self refers to the actual step (and the class of course), but log is coming from where? I was not able to find the declaration or assignment statement for logs.....it should come from the step itself right? In this case what kind of declaration should i do to get the object "log"?
> 
> I have done this so far to get the errors:
> 
> class BetterCompile(ShellCommand):
> command = ["xcodebuild", "-configuration", "Debug"]
> 
> def createSummary(self, log):
> # 'log' is the stdio LogFile, there might be others
> text = StringIO(log.getText())
> warnings = [line for line in text.readlines()
> if line.startswith("warning:")]
> self.addCompleteLog('warnings', "".join(warnings))
> 
> f.addStep(BetterCompile)
> 
> and when i do the checkconfig i don't get any error, but when i run the step it gives me an exception
> 
> 2011-02-07 13:04:39-0800 [-] BuildStep.failed, traceback follows
> 2011-02-07 13:04:39-0800 [-] Unhandled Error
> Traceback (most recent call last):
>  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 186, in addCallbacks
>   self._runCallbacks()
>  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 328, in _runCallbacks
>   self.result = callback(self.result, *args, **kw)
>  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 243, in callback
>   self._startRunCallbacks(result)
>  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 312, in _startRunCallbacks
>   self._runCallbacks()
> --- <exception caught here> ---
>  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py", line 328, in _runCallbacks
>   self.result = callback(self.result, *args, **kw)
>  File "/Library/Python/2.6/site-packages/buildbot-0.8.3p1-py2.6.egg/buildbot/process/buildstep.py", line 1093, in <lambda>
>   d.addCallback(lambda res: self.createSummary(cmd.logs['stdio']))
>  File "/masterbot/master.cfg", line 140, in createSummary
>   text = StringIO(log.getText())
> exceptions.NameError: global name 'StringIO' is not defined
> 
> 
> Am I missing something or this example is not working? Is there any other way to just get the stdio file output and grep it for errors and warning, and then put the output in an object that can be accessed by the mail notifier and the waterfall page?
> 
> Thanks!
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110207/6e5dbae1/attachment.html>