[Buildbot-commits] [Buildbot] #2234: have a better-defined approach to mocking

[Buildbot trac]

#2234: have a better-defined approach to mocking
------------------------+---------------------
Reporter:  dustin       |       Owner:  dustin
    Type:  enhancement  |      Status:  closed
Priority:  major        |   Milestone:  0.9.+
 Version:  0.8.5        |  Resolution:  fixed
Keywords:  tests        |
------------------------+---------------------
Changes (by dustin):

 * status:  new => closed
 * resolution:   => fixed


Old description:

> This year's testing track at !PyCon had a lot of content about hitting
> the right approach with mocks -- and the pains of not doing so.
>
> One major point was to make your mocks authoritative (one or zero mocks
> for any system), narrow (minimal implementation), and isolated (don't
> depend on the real system).
>
> Most of the mocks used in Buildbot are already authoritative -- the stuff
> in `master/buidlbot/test/fake` in particular, but even some of the mocks
> in individual test files are unique.  There are a lot of status and
> process class mocks, thought, that are not authoritative.
>
> The DB mocks, at least, are narrow - they are pretty poor implementations
> of the API, in fact.  And they are quite well isolated.
>
> The path from here is:
>  * state this as policy
>    * check patches against it
>    * fix existing violations
>  * test the mocks
>    * mocks should pass the same tests that the system they're mocking
> pass
>      * especially for DB API
>    * where this isn't practical, using introspection to compare function
> signatures gets 80% of the win

New description:

 This year's testing track at !PyCon had a lot of content about hitting the
 right approach with mocks -- and the pains of not doing so.

 One major point was to make your mocks authoritative (one or zero mocks
 for any system), narrow (minimal implementation), and isolated (don't
 depend on the real system).

 Most of the mocks used in Buildbot are already authoritative -- the stuff
 in `master/buidlbot/test/fake` in particular, but even some of the mocks
 in individual test files are unique.  There are a lot of status and
 process class mocks, thought, that are not authoritative.

 The DB mocks, at least, are narrow - they are pretty poor implementations
 of the API, in fact.  And they are quite well isolated.

 The path from here is:
  * state this as policy
    * check patches against it
    * fix existing violations
  * test the mocks
    * mocks should pass the same tests that the system they're mocking pass
      * especially for DB API
    * where this isn't practical, using introspection to compare function
 signatures gets 80% of the win

--

Comment:

 This is sufficiently internalized now that there's no action on this bug.

-- 
Ticket URL: <http://trac.buildbot.net/ticket/2234#comment:4>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation