[Buildbot-devel] Marking a build step as failed unless the STDOUT contains a magic string.
matisse at matisse.net
Tue Jan 22 22:33:49 UTC 2008
On Jan 22, 2008, at 9:29 AM, Scott Lawrence wrote:
> On Tue, 2008-01-22 at 18:17 +0100, Daniel Svensson wrote:
>> On Jan 22, 2008 5:10 PM, Scott Lawrence <slawrence at pingtel.com>
>>> It's not very robust to look for magic strings. It's much better to
>>> just make sure that your makefile is written so that 'make' will
>>> a failure status unless all tests pass.
>> How do you show how many tests passed/failed? What if you want to
>> extend the mechanism to keep track of testcases that doesn't meet the
>> required coverage goal?
> I didn't mean to imply that you shouldn't keep track of the numbers -
> just that looking for strings in the output is easy to break.
> With some work, you can get the makefile itself to enforce nearly any
> rule you can dream up to make the pass/fail decision, but it's easiest
> if you just get that decision as an exit status from 'make'.
The make target in question runs tests and then submits the results to
a server (http://smoke.parrotcode.org/smoke/) - the target "succeeds"
if the submission of data succeeds, regardless of the tests themselves
passing or failing.
So unless I want to have a separate step that just runs 'make
test' (which duplicates running the tests), looking for the magic
string is the best I can do for now.
Matisse Enzer <matisse at matisse.net>
http://www.matisse.net/ - http://www.eigenstate.net/
More information about the devel