[Buildbot-devel] Parse the stdio file for errors and warnings and add them to the waterfall page and on an email formatter

Tue Feb 8 00:20:16 UTC 2011

The vstudio.py (and others) present stdio, warnings, and error logs.

You can add any number of named logs using text you have built up, in the
createSummary() method.

This should all be done in one Step, since (I'm guessing) you want to
present summaries of the results of a one compile?

xcode understands python files.

I have found the best way to work with buildbot is to start with a very
simple modification of an existing class, that executes a simple dummy
compile, and make small changes iteratively.

good luck!

On Mon, Feb 7, 2011 at 3:12 PM, Fulvio Cervone <fcervone at me.com> wrote:

> Thanks Alexander and Philippe
>
> I've found (not without a lot of perils) the class called* *
> WarningCountingShellCommand;
>
>
> http://buildbot.net/buildbot/docs/0.8.3/reference/buildbot.steps.shell.WarningCountingShellCommand-class.html
>
> Once found the class, I tried to apply it instead of the ShellCommand, and
> actually I can get the warnings and errors (just have to modify the warning
> pattern)
>
> Now if i wanna get 2 logs(warning and errors) should i run 2 steps or is
> there a way to specify more than one log, with different name? (as now the
> Waterfall page shows "warnings (n)" independently from what i use as search
> pattern)
>
> BTW I use Xcode, is the vstudio.py still useful to get the gist of how this
> works?
>
>
>
>
> On Feb 7, 2011, at 3:01 PM, Philippe McLean wrote:
>
> The visual studio compile steps in buildbot/steps/vstudio.py also have an
> example of doing this.
>
> On Mon, Feb 7, 2011 at 1:55 PM, Alexander O'Donovan-Jones <
> Alexander at ccpgames.com> wrote:
>
>> Have you looked into the WarningCountingShellCommand class? It does pretty
>> much what you're after.
>> ________________________________________
>> From: Fulvio Cervone [fcervone at me.com]
>> Sent: 07 February 2011 21:16
>> To: buildbot-devel list
>> Subject: [Buildbot-devel] Parse the stdio file for errors and warnings and
>> add them to the waterfall page and on an email formatter
>>
>> I am getting kinda swamped in this.....
>>
>> I've checked the manual and the mailing list, and seems that there is no
>> quick way to do a grep on the stdio and just get the output and show it on
>> the waterfall page, so from what i have seen, you have to use logfiles and
>> LogObservers.
>>
>> What i want to achieve is to get the errors and warnings from the build
>> process and put them in 2 files on the waterfall page, and in case of errors
>> to get also the output of the file into my customized email formatter.
>>
>> I've found this example here on a post in the mailing list history:
>>
>> === 2
>>
>> class BetterCompile(Compile):
>> command = ["make", "all"]
>>
>> def createSummary(self, log):
>> # 'log' is the stdio LogFile, there might be others
>> text = StringIO(log.getText())
>> warnings = [line for line in text.readlines()
>> if line.startswith("warning:")]
>> self.addCompleteLog('warnings', "".join(warnings))
>>
>> f.addStep(BetterCompile)
>>
>> This would work for what i need (just have to do another parsing for the
>> errors and i am done basically), but I am not clear how this works...
>>
>> 1) the class is created inheriting from Compile, I do not use compile but
>> ShellCommand; may I do the same using ShellCommand? (changing of course the
>> command)
>>
>> 2) when the def createSummary is done, self refers to the actual step (and
>> the class of course), but log is coming from where? I was not able to find
>> the declaration or assignment statement for logs.....it should come from the
>> step itself right? In this case what kind of declaration should i do to get
>> the object "log"?
>>
>> I have done this so far to get the errors:
>>
>> class BetterCompile(ShellCommand):
>> command = ["xcodebuild", "-configuration", "Debug"]
>>
>> def createSummary(self, log):
>> # 'log' is the stdio LogFile, there might be others
>> text = StringIO(log.getText())
>> warnings = [line for line in text.readlines()
>> if line.startswith("warning:")]
>> self.addCompleteLog('warnings', "".join(warnings))
>>
>> f.addStep(BetterCompile)
>>
>> and when i do the checkconfig i don't get any error, but when i run the
>> step it gives me an exception
>>
>> 2011-02-07 13:04:39-0800 [-] BuildStep.failed, traceback follows
>> 2011-02-07 13:04:39-0800 [-] Unhandled Error
>> Traceback (most recent call last):
>>  File
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
>> line 186, in addCallbacks
>>   self._runCallbacks()
>>  File
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
>> line 328, in _runCallbacks
>>   self.result = callback(self.result, *args, **kw)
>>  File
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
>> line 243, in callback
>>   self._startRunCallbacks(result)
>>  File
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
>> line 312, in _startRunCallbacks
>>   self._runCallbacks()
>> --- <exception caught here> ---
>>  File
>> "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/internet/defer.py",
>> line 328, in _runCallbacks
>>   self.result = callback(self.result, *args, **kw)
>>  File
>> "/Library/Python/2.6/site-packages/buildbot-0.8.3p1-py2.6.egg/buildbot/process/buildstep.py",
>> line 1093, in <lambda>
>>   d.addCallback(lambda res: self.createSummary(cmd.logs['stdio']))
>>  File "/masterbot/master.cfg", line 140, in createSummary
>>   text = StringIO(log.getText())
>> exceptions.NameError: global name 'StringIO' is not defined
>>
>>
>> Am I missing something or this example is not working? Is there any other
>> way to just get the stdio file output and grep it for errors and warning,
>> and then put the output in an object that can be accessed by the mail
>> notifier and the waterfall page?
>>
>> Thanks!
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
>> Pinpoint memory and threading errors before they happen.
>> Find and fix more than 250 security defects in the development cycle.
>> Locate bottlenecks in serial and parallel code that limit performance.
>> http://p.sf.net/sfu/intel-dev2devfeb
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110207/f234e2b0/attachment.html>