[Buildbot-devel] Marking a build step as failed unless the STDOUT contains a magic string.

[Scott Lawrence]

On Tue, 2008-01-22 at 18:17 +0100, Daniel Svensson wrote:
> On Jan 22, 2008 5:10 PM, Scott Lawrence <slawrence at pingtel.com> wrote:
> > It's not very robust to look for magic strings.  It's much better to
> > just make sure that your makefile is written so that 'make' will return
> > a failure status unless all tests pass.
> 
> How do you show how many tests passed/failed? What if you want to
> extend the mechanism to keep track of testcases that doesn't meet the
> required coverage goal?

I didn't mean to imply that you shouldn't keep track of the numbers -
just that looking for strings in the output is easy to break.

With some work, you can get the makefile itself to enforce nearly any
rule you can dream up to make the pass/fail decision, but it's easiest
if you just get that decision as an exit status from 'make'.

-- 
Scott Lawrence  tel:+1.781.229.0533;ext=162 or sip:slawrence at pingtel.com
  sipXecs project coordinator - SIPfoundry http://www.sipfoundry.org/sipXecs
  CTO, Voice Solutions   - Bluesocket Inc. http://www.bluesocket.com/ 
                                           http://www.pingtel.com/