[Buildbot-commits] [Buildbot] #2269: test real RemoteCommands and FakeRemoteCommands with the same tests (was: Rewrite buildstep tests against real RemoteCommands.)

[Buildbot]

#2269: test real RemoteCommands and FakeRemoteCommands with the same tests
------------------------+--------------------
Reporter:  tom.prince   |       Owner:
    Type:  enhancement  |      Status:  new
Priority:  major        |   Milestone:  0.8.+
 Version:  0.8.6p1      |  Resolution:
Keywords:  tests        |
------------------------+--------------------

Comment (by dustin):

 Summary adjusted with what I think is a better solution here -- test that
 the fakes and the real implementations pass the same tests -
 http://pyvideo.org/video/629/stop-mocking-start-testing

 From discussion in irc, though, it sounds like this case was a little more
 tricky -- an invalid function keyword argument was added to
 !FakeRemoteCommand that was not present in !RemoteCommand.  I think there
 are three ways to uncover problems like this, and we should aim to do all
 three:

  * test fakes and real objects with the same tests
    * use introspection to make assertions about function/method arguments
  * once this is in place and known, code review should look for
 parallelism in changes to real and fake objects
  * light use of integration testing as a fallback

 I'm not a big fan of heavy-duty integration testing, as it's *very*
 difficult to hit every possibility.  In this case, a few integration tests
 running e.g., Git steps against a fake slave may not have triggered this
 particular error.

-- 
Ticket URL: <http://trac.buildbot.net/ticket/2269#comment:3>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation