[Buildbot-devel] Problem with SVNPoller on windows
Oscar Lindberg
oscar.lindberg at todos.se
Mon Sep 28 09:03:10 UTC 2009
Subject: Problem with SVNPoller on windows
Hi,
I'm new to buildbot and also this mailing list, so I hope I'm not doing
something wrong by posting this here.
Currently I'm setting up my first buildbot project. I've had some problems
with SVNPoller on windows, and I had to resort to a particularly ugly
work-around to make it work (hacking twisted). My hope is that someone could
give me a better alternative, or that this post could result in some patch of
buildbot (or twisted) in the future.
OK, I'm running:
--------------------------------------------------
C:\Documents and Settings\todos>buildbot --version
Buildbot version: 0.7.11p3
Twisted version: 8.2.0
C:\Documents and Settings\todos>python --version
Python 2.6.2
--------------------------------------------------
This results in some warnings at startup, Python 2.6 does not like the modules
that twisted 8.2.0 is using, but that is probably not the problem:
-------------------------------------------------------------------------------
C:\Documents and Settings\todos\My Documents\buildbot_master\x>buildbot start .
C:\Python26\lib\site-packages\twisted\persisted\sob.py:12: DeprecationWarning:
the md5 module is deprecated; use hashlib instead
import os, md5, sys
C:\Python26\lib\site-packages\twisted\python\filepath.py:12: DeprecationWarning
: the sha module is deprecated; use the hashlib module instead
import sha
C:\Python26\lib\site-packages\twisted\mail\smtp.py:10: DeprecationWarning: the
MimeWriter module is deprecated; use the email package instead
import MimeWriter, tempfile, rfc822
-------------------------------------------------------------------------------
The problem is my SVNPoller:
from master.cfg:
------------------------------------------------
from buildbot.changes.svnpoller import SVNPoller
svn_poller = SVNPoller(
svnurl=svn_url,
pollinterval=5*60, # seconds
histmax=10,
)
c['sources'] = [ svn_poller ]
------------------------------------------------
When I start, the SVNPoller will throw an exception, and it will not run.
In twistd.log:
------------------------------------------------------------------------------
2009-09-28 13:25:56+0800 [-] SVNPoller polling
2009-09-28 13:25:56+0800 [-] SVNPoller getProcessOutput: svn with args ['info',
'--xml', '--non-interactive', 'svn+ssh://x.se/y/z']
2009-09-28 13:25:56+0800 [-] Unhandled error in Deferred:
2009-09-28 13:25:56+0800 [-] Unhandled Error
Traceback (most recent call last):
File "C:\Python26\lib\site-packages\twisted\internet\base.py",
line 1137, in mainLoop
self.runUntilCurrent()
File "C:\Python26\lib\site-packages\twisted\internet\base.py",
line 757, in runUntilCurrent
call.func(*call.args, **call.kw)
File "C:\Python26\lib\site-packages\twisted\internet\task.py",
line 83, in start
self()
File "C:\Python26\lib\site-packages\twisted\internet\task.py",
line 114, in __call__
d = defer.maybeDeferred(self.f, *self.a, **self.kw)
--- <exception caught here> ---
File "C:\Python26\lib\site-packages\twisted\internet\defer.py",
line 106, in maybeDeferred
result = f(*args, **kw)
File "C:\Python26\lib\site-packages\buildbot\changes\svnpoller.py",
line 258, in checksvn
d = self.get_root()
File "C:\Python26\lib\site-packages\buildbot\changes\svnpoller.py",
line 283, in get_root
d = self.getProcessOutput(args)
File "C:\Python26\lib\site-packages\buildbot\changes\svnpoller.py",
line 274, in getProcessOutput
d = utils.getProcessOutput(self.svnbin, args, {})
File "C:\Python26\lib\site-packages\twisted\internet\utils.py",
line 124, in getProcessOutput
reactor)
File "C:\Python26\lib\site-packages\twisted\internet\utils.py",
line 25, in _callProtocolWithDeferred
reactor.spawnProcess(p, executable, (executable,)+tuple(args), env,
path)
File "C:\Python26\lib\site-packages\twisted\internet\
win32eventreactor.py", line 234, in spawnProcess
return Process(self, processProtocol, executable, args, env, path)
File "C:\Python26\lib\site-packages\twisted\internet\
_dumbwin32proc.py", line 182, in __init__
raise OSError(pwte)
exceptions.OSError: (2, 'CreateProcess', 'The system cannot find the
file specified.')
-------------------------------------------------------------------------------
By adding some logging and reading some source code, I found out that the call
throwing the OSError is equivalent to:
-------------------------------------------------------------------------------
command = 'svn'
cmdline = 'svn info --xml --non-interactive svn+ssh://x.se/y/z'
win32process.CreateProcess(command, cmdline, None, None, 1, 0, env, None,
StartupInfo)
-------------------------------------------------------------------------------
If I run that separately, I get the same exception (exceptions.OSError: (2,
'CreateProcess', 'The system cannot find the file specified.')).
I read about the CreateProcess at http://docs.activestate.com/activepython/
2.4/pywin32/win32process__CreateProcess_meth.html:
-------------------------------------------------------------------------------
win32process.CreateProcess
PyHANDLE, PyHANDLE, int, int = CreateProcess(appName, commandLine ,
processAttributes , threadAttributes , bInheritHandles , dwCreationFlags ,
newEnvironment , currentDirectory , startupinfo )
Creates a new process and its primary thread. The new process executes the
specified executable file.
Parameters
appName : string
name of executable module, or None
commandLine : string
command line string, or None
-------------------------------------------------------------------------------
I thought that I should try to use None as the appName, since the command name
is already in the commandLine.
That made the exception go away. "Svn info" now runs.
So I added my one-line-hack at the beginning of
Process.__init__ in twisted\internet\_dumbwin32proc.py:
command = None if command == 'svn' else command
Now my SVNRunner started, and it works like a charm.
I could not move my hack to the buildbot source, because executable needs to
be a string and I need to pass None.
I might be the first one to stumble upon the problem because I am running the
buildbot master on XP, I am using SVNPoller instead of a svn hook.
So my questions are:
Is there a better way to make this work? (I really hope so.)
Is this a bug that I should report?
If it is a bug, is it a bug in buildbot or in twisted?
Best Regards,
Oscar Lindberg
More information about the devel
mailing list