[Buildbot-devel] differences in test coverage from 0.7.x to 0.8.2?

[Axel Hecht]

I've spun a quick coverage analysis with
http://nedbatchelder.com/code/coverage/, for some recent git version, I ran

PYTHONPATH=master:slave coverage run --include='*build*'
~/envs/twisted/bin/trial buildbot.test buildslave.test
coverage html --include='*build*'

and for my 0.7.12 fork,

PYTHONPATH=. coverage run --include='*buildbot*' ~/envs/twisted/bin/trial
buildbot.test
# same html command.

The output is on

http://people.mozilla.org/~axel/buildbot-coverage/0.8.2/ and
http://people.mozilla.org/~axel/buildbot-coverage/0.7.12/

YMMV

Axel

PS: Dummy is at
https://github.com/buildbot/buildbot/blob/master/master/buildbot/steps/dummy.py#L8
.

2010/11/21 Dustin J. Mitchell <dustin at v.igoro.us>

> On Fri, Nov 19, 2010 at 8:03 PM, Axel Hecht <l10n.moz at googlemail.com>
> wrote:
> > I've been having day-dreams about stuff on top of buildbot, mostly based
> on
> > tests I knew existed on 0.7.x.
> >
> > Now, I've tried to find tests like those on 0.8.2, and I'm not
> successful.
> > Like, no tests mentioning dummy.Dummy or Triggerable.
> >
> > Did those tests go anywhere? And if so, is there a reason they did?
>
> Yes, about a year ago we blew away all of the flaky, unreliable tests,
> because they were causing more harm than good:
>  * downstream packagers were delaying releases due to failing tests,
> and didn't appreciate the "oh, yeah, those tests are flaky" excuse
>  * contributors were confused that tests were flaking when they added new
> code
>  * new contributions would break unrelated tests that made invalid
> assumptions
>  * the tests were not at all isolated, so "fixing" = "completely rewriting"
>
> This was a controversial decision, but I think that history has shown
> it to be a net positive.  As new, better-isolated code is written,
> it's getting new tests.  We've also built some better frameworks to
> allow more natural isolation of units for testing.
>
> I'm not surprised there aren't tests for dummy.Dummy, since that's not
> in the codebase :)
>
> Dustin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20101121/02209271/attachment.html>