[Buildbot-devel] git step fails to remove old build directory, causing source to be copied to wrong directory

[Dan Kegel]

I'm using buildbot-slave-0.8.7 (out of inertia), with the git step like:

        factory.addStep(Git(repourl=repourl, mode='full',
submodules=True, method='copy', branch=sbranch,
getDescription={'tags':True}))

Sometimes a previous build leaves some whacky process running,
so when the next git checkout tries to remove the old build directory,
it fails with

rm: cannot remove
'/home/buildbot/slave-state/cygwin/g-speak-yobuild10-win7/mzreach-webrtc-win7-msvc2013-master/build/webrtc/trunk/chromium/_gclient_src_4cokd8/.git/objects/pack/tmp_pack_Khspxc':
Device or resource busy

Buildbot then does a chmod (in hopes of fixing the failure), but of
course the 2nd rm still fails, since it wasn't a permission problem.

That second failure should trigger a build failure, but doesn't.  As a result,
the following cp creates a source subdirectory in the old build directory (!)
which is rather confusing.

It kind of looks like the source tries to abort if the 2nd rm fails;
not sure why
that doesn't happen.

I should probably have the bot reboot after each build to avoid the problem.

But on the buildbot side, assuming this hasn't already been fixed:
I guess the question is, how does one add a unit test for this?
I imagine add one to buildslave/test/unit/test_commands_git.py
that uses a mock /bin/rm that always fails?
- Dan