[Buildbot-devel] Problem with SVNPoller on windows

Mon Sep 28 09:29:13 UTC 2009

I had similar problem, which I solved like this

svn_poller = SVNPoller(
                    svnurl=source_code_svn_url,
                    pollinterval=15*60, # seconds
                    histmax=10,
                    svnbin="c:\svn.exe") 

Note: in svnbin I also have problem if I use path which contains spaces
it did not work, but with out spaces it work as it should.

I hope this info will help you to resolve you problem.

Bye Tomaz

-----Original Message-----
From: Oscar Lindberg [mailto:oscar.lindberg at todos.se] 
Sent: Monday, September 28, 2009 11:03 AM
To: buildbot-devel at lists.sourceforge.net
Subject: [Buildbot-devel] Problem with SVNPoller on windows

Subject: Problem with SVNPoller on windows

Hi, 

I'm new to buildbot and also this mailing list, so I hope I'm not doing
something wrong by posting this here. 

Currently I'm setting up my first buildbot project. I've had some
problems with SVNPoller on windows, and I had to resort to a
particularly ugly work-around to make it work (hacking twisted). My hope
is that someone could give me a better alternative, or that this post
could result in some patch of buildbot (or twisted) in the future. 

OK, I'm running: 

--------------------------------------------------
C:\Documents and Settings\todos>buildbot --version Buildbot version:
0.7.11p3 Twisted version: 8.2.0

C:\Documents and Settings\todos>python --version Python 2.6.2
--------------------------------------------------

This results in some warnings at startup, Python 2.6 does not like the
modules that twisted 8.2.0 is using, but that is probably not the
problem: 

------------------------------------------------------------------------
-------
C:\Documents and Settings\todos\My Documents\buildbot_master\x>buildbot
start .
C:\Python26\lib\site-packages\twisted\persisted\sob.py:12:
DeprecationWarning: 
the md5 module is deprecated; use hashlib instead
  import os, md5, sys
C:\Python26\lib\site-packages\twisted\python\filepath.py:12:
DeprecationWarning
: the sha module is deprecated; use the hashlib module instead
  import sha
C:\Python26\lib\site-packages\twisted\mail\smtp.py:10:
DeprecationWarning: the MimeWriter module is deprecated; use the email
package instead
  import MimeWriter, tempfile, rfc822
------------------------------------------------------------------------
-------

The problem is my SVNPoller: 

from master.cfg: 
------------------------------------------------
from buildbot.changes.svnpoller import SVNPoller svn_poller = SVNPoller(
                   svnurl=svn_url,
                   pollinterval=5*60, # seconds
                   histmax=10,
 )
c['sources'] = [ svn_poller ]
------------------------------------------------

When I start, the SVNPoller will throw an exception, and it will not
run. 

In twistd.log:
------------------------------------------------------------------------
------
2009-09-28 13:25:56+0800 [-] SVNPoller polling
2009-09-28 13:25:56+0800 [-] SVNPoller getProcessOutput: svn with args
['info',  '--xml', '--non-interactive', 'svn+ssh://x.se/y/z']
2009-09-28 13:25:56+0800 [-] Unhandled error in Deferred:
2009-09-28 13:25:56+0800 [-] Unhandled Error
	Traceback (most recent call last):
	  File "C:\Python26\lib\site-packages\twisted\internet\base.py",
line 1137, in mainLoop
	    self.runUntilCurrent()
	  File "C:\Python26\lib\site-packages\twisted\internet\base.py",
line 757, in runUntilCurrent
	    call.func(*call.args, **call.kw)
	  File "C:\Python26\lib\site-packages\twisted\internet\task.py",
line 83, in start
	    self()
	  File "C:\Python26\lib\site-packages\twisted\internet\task.py",
line 114, in __call__
	    d = defer.maybeDeferred(self.f, *self.a, **self.kw)
	--- <exception caught here> ---
	  File
"C:\Python26\lib\site-packages\twisted\internet\defer.py",
line 106, in maybeDeferred
	    result = f(*args, **kw)
	  File
"C:\Python26\lib\site-packages\buildbot\changes\svnpoller.py",
line 258, in checksvn
	    d = self.get_root()
	  File
"C:\Python26\lib\site-packages\buildbot\changes\svnpoller.py",
line 283, in get_root
	    d = self.getProcessOutput(args)
	  File
"C:\Python26\lib\site-packages\buildbot\changes\svnpoller.py",
line 274, in getProcessOutput
	    d = utils.getProcessOutput(self.svnbin, args, {})
	  File
"C:\Python26\lib\site-packages\twisted\internet\utils.py",
line 124, in getProcessOutput
	    reactor)
	  File
"C:\Python26\lib\site-packages\twisted\internet\utils.py",
line 25, in _callProtocolWithDeferred
	    reactor.spawnProcess(p, executable,
(executable,)+tuple(args), env,
path)
	  File "C:\Python26\lib\site-packages\twisted\internet\
win32eventreactor.py", line 234, in spawnProcess
	    return Process(self, processProtocol, executable, args, env,
path)
	  File "C:\Python26\lib\site-packages\twisted\internet\
_dumbwin32proc.py", line 182, in __init__
	    raise OSError(pwte)
	exceptions.OSError: (2, 'CreateProcess', 'The system cannot find
the file specified.')
------------------------------------------------------------------------
-------

By adding some logging and reading some source code, I found out that
the call throwing the OSError is equivalent to: 

------------------------------------------------------------------------
-------
command = 'svn'
cmdline = 'svn info --xml --non-interactive svn+ssh://x.se/y/z'
win32process.CreateProcess(command, cmdline, None, None, 1, 0, env,
None,
StartupInfo)
------------------------------------------------------------------------
-------

If I run that separately, I get the same exception (exceptions.OSError:
(2, 'CreateProcess', 'The system cannot find the file specified.')). 

I read about the CreateProcess at
http://docs.activestate.com/activepython/
2.4/pywin32/win32process__CreateProcess_meth.html:

------------------------------------------------------------------------
-------
win32process.CreateProcess
PyHANDLE, PyHANDLE, int, int = CreateProcess(appName, commandLine ,
processAttributes , threadAttributes , bInheritHandles , dwCreationFlags
, newEnvironment , currentDirectory , startupinfo )

Creates a new process and its primary thread. The new process executes
the specified executable file.

Parameters
  appName : string
    name of executable module, or None
  commandLine : string
    command line string, or None
------------------------------------------------------------------------
-------

I thought that I should try to use None as the appName, since the
command name 
is already in the commandLine. 

That made the exception go away. "Svn info" now runs.

So I added my one-line-hack at the beginning of 
Process.__init__ in twisted\internet\_dumbwin32proc.py:
        command = None if command == 'svn' else command

Now my SVNRunner started, and it works like a charm. 

I could not move my hack to the buildbot source, because executable
needs to 
be a string and I need to pass None. 

I might be the first one to stumble upon the problem because I am
running the 
buildbot master on XP, I am using SVNPoller instead of a svn hook. 

So my questions are: 
Is there a better way to make this work? (I really hope so.) 
Is this a bug that I should report? 
If it is a bug, is it a bug in buildbot or in twisted? 

Best Regards, 
  Oscar Lindberg

------------------------------------------------------------------------
------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and
stay 
ahead of the curve. Join us from November 9-12, 2009. Register
now!
http://p.sf.net/sfu/devconf
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/buildbot-devel