[Buildbot-devel] Exception when performing full SVN update on windows slave

[Jean-Marc Dressler]

Hello,

First I want to mention that I have tried to report this problem
through trac, but my ticket always gets rejected by the trac spam
filter.

I get the following exception in my buildslave (running on a
french windows XP system) each time it tries to perform a 'full'
update ('incremental' update works fine) of its SVN working copy:

2011-10-24 21:15:46+0200 [Broker,client]  startCommand:rmdir [id
3]
2011-10-24 21:15:46+0200 [Broker,client] DIRECTORY
C:\buildbot\slave\unattendedtests\ArkiScripts/arkitest\test.log
2011-10-24 21:15:46+0200 [Broker,client]
SlaveBuilder.commandFailed
<buildslave.commands.fs.RemoveDirectory instance at 0x010E8AD0>
2011-10-24 21:15:46+0200 [Broker,client] Unhandled Error
Traceback (most recent call last):
 File
"c:\Python27\lib\site-packages\buildbot_slave-0.8.5-py2.7.egg\bui
ldslave\bot.py", line 141, in remote_startCommand
   d = self.command.doStart()
 File
"c:\Python27\lib\site-packages\buildbot_slave-0.8.5-py2.7.egg\bui
ldslave\commands\base.py", line 145, in doStart
   d = defer.maybeDeferred(self.start)
 File "c:\Python27\lib\site-packages\twisted\internet\defer.py",
line 133, in maybeDeferred
   result = f(*args, **kw)
 File "c:\Python27\lib\site-packages\twisted\internet\defer.py",
line 964, in unwindGenerator
   return _deferGenerator(f(*args, **kwargs), Deferred())
--- <exception caught here> ---
 File "c:\Python27\lib\site-packages\twisted\internet\defer.py",
line 866, in _deferGenerator
   result = g.next()
 File
"c:\Python27\lib\site-packages\buildbot_slave-0.8.5-py2.7.egg\bui
ldslave\commands\fs.py", line 73, in start
   wfd = defer.waitForDeferred(self.removeSingleDir(dirnames))
 File
"c:\Python27\lib\site-packages\buildbot_slave-0.8.5-py2.7.egg\bui
ldslave\commands\fs.py", line 85, in removeSingleDir
   utils.rmdirRecursive(self.dir)
 File
"c:\Python27\lib\site-packages\buildbot_slave-0.8.5-py2.7.egg\bui
ldslave\commands\utils.py", line 72, in rmdirRecursive
   log.msg("rmdirRecursive: unable to listdir %s (%s). Trying to
remove like a dir" % (dir, e.strerror))
exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte
0xe9 in position 11: ordinal not in range(128)

I have tracked the cause of the exception to
buildslave.commands.utils:

(1) The first issue is that the error message (`strerror`)
returned with the `WindowsError` exception can contain non ascii
characters (on my french windows XP for example), which causes an
uncaught exception.

(2) But the main issue is that the rmdirRecursive() function does
not check if its argument is a file or not. When it is a file a
call to os.listdir fails which triggers the exception with the
unicode character problem described in (1).

Below is the fix I use for problem (2) (in the module
buildslave.commands.utils). Note that I did not try to fix (1)
but I will be happy to help with testing.

if runtime.platformType  == 'win32':
    def rmdirRecursive(dir):
        """This is a replacement for shutil.rmtree that works
better under
        windows. Thanks to Bear at the OSAF for the code."""
        if not os.path.exists(dir):
            return

        if os.path.islink(dir):
            os.remove(dir)
            return

        # Verify the directory is read/write/execute for the
current user
        os.chmod(dir, 0700)

        #### Bug fix: check if dir is a file ##################
        if os.path.isfile(dir):
            os.remove(dir)
            return
        #######################################################

Thanks,

--
Jean-Marc Dressler
-- 
Jean-Marc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20111208/a383a0be/attachment.html>