[Buildbot-devel] Recommendations for client / server testing

[Dustin J. Mitchell]

On Wed, Jun 2, 2010 at 12:10 AM, Brad Hards <bradh at frogmouth.net> wrote:
> I've been using buildbot to test openchange (http://www.openchange.org) for a
> while, and it mostly works pretty well for us for client-side testing (against
> a running Exchange server).

Great!  Please add it to the SuccessStories on buildbot.net?

> We're about to move into testing of the server side of openchange. Essentially
> I want to build the source, configure the server, start the server, wait a
> little while (or for a certain debug output line) and then run my client
> tests. When all the tests are done, kill the server and wait for the next
> checkin.
>
> Any suggestions, tips or tricks for how to best approach this?

The danger is, of course, that the server continues running forever if
something goes wrong.

We've dealt with this in testing Amanda by adding some code to the
daemon that kills the daemon if its stdin closes.  Then we have a
parent process that runs the daemon as a child process with stdin held
open, and then runs all of the tests.  If that parent process dies for
any reason, the OS closes stdin and the daemon exits (unless it's
hung, of course).

You can also use the stdio channel to allow the daemon to signal its
readiness to begin the tests, rather than waiting a little while.  In
my experience, doing testing that involves time intervals leads to
very fragile tests.

Dustin

-- 
Open Source Storage Engineer
http://www.zmanda.com